-- Logs begin at Tue 2022-05-10 19:16:42 UTC, end at Tue 2022-05-10 21:02:48 UTC. -- May 10 19:56:31.408502 dv-cinder-828805-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 10 19:56:31.416453 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 10 19:56:31.422107 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue May 10 19:56:31 2022] *** May 10 19:56:31.422107 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 10 19:56:31.422107 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 10 19:56:31.422107 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: nodename: dv-cinder-828805-2-iscsi May 10 19:56:31.422107 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: machine: x86_64 May 10 19:56:31.422107 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: clock source: unix May 10 19:56:31.422107 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: pcre jit disabled May 10 19:56:31.422107 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: detected number of CPU cores: 4 May 10 19:56:31.422107 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: current working directory: / May 10 19:56:31.422107 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: detected binary path: /usr/bin/uwsgi-core May 10 19:56:31.422107 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: your processes number limit is 47680 May 10 19:56:31.422107 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: your memory page size is 4096 bytes May 10 19:56:31.422107 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: detected max file descriptor number: 2048 May 10 19:56:31.422107 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: lock engine: pthread robust mutexes May 10 19:56:31.422107 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: thunder lock: enabled May 10 19:56:31.422107 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 10 19:56:31.422107 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 10 19:56:31.467504 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: Python main interpreter initialized at 0x5582e998f9c0 May 10 19:56:31.467504 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: python threads support enabled May 10 19:56:31.467504 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: your server socket listen backlog is limited to 100 connections May 10 19:56:31.467504 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: your mercy for graceful operations on workers is 90 seconds May 10 19:56:31.467922 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: mapped 403077 bytes (393 KB) for 2 cores May 10 19:56:31.468001 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: *** Operational MODE: preforking *** May 10 19:56:31.468262 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: *** uWSGI is running in multiple interpreter mode *** May 10 19:56:31.468262 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: spawned uWSGI master process (pid: 80501) May 10 19:56:31.468467 dv-cinder-828805-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 10 19:56:31.470149 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: spawned uWSGI worker 1 (pid: 80515, cores: 1) May 10 19:56:31.470149 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: spawned uWSGI worker 2 (pid: 80516, cores: 1) May 10 19:56:31.470149 dv-cinder-828805-2-iscsi devstack@keystone.service[80501]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 10 19:56:37.217573 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 10 19:56:37.217573 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: from cryptography.utils import int_from_bytes May 10 19:56:37.218393 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 10 19:56:37.218393 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: from cryptography.utils import int_from_bytes May 10 19:56:37.218849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 10 19:56:37.218849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: from cryptography.utils import int_from_bytes May 10 19:56:37.219042 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 10 19:56:37.219042 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: from cryptography.utils import int_from_bytes May 10 19:56:37.277607 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 10 19:56:37.277786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 10 19:56:37.277862 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 10 19:56:37.277936 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 10 19:56:37.278003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] command line args: [] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 10 19:56:37.278072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] command line args: [] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 10 19:56:37.278332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 10 19:56:37.278461 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 10 19:56:37.278547 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ================================================================================ {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 10 19:56:37.278627 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ================================================================================ {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 10 19:56:37.278711 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] admin_token = **** {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.278783 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] admin_token = **** {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.278858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] config_dir = [] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.278992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] config_dir = [] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.279072 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.279181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.279268 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] config_source = [] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.279346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] config_source = [] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.279425 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] debug = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.279529 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] debug = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.279615 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.279807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.279892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.279956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.280118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.280369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.280456 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.280616 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.280699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.280843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] list_limit = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.280921 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.281023 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] log_config_append = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.281103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] list_limit = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.281203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.281282 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] log_config_append = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.281375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] log_dir = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.281452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.281545 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] log_file = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.281623 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] log_dir = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.281738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] log_file = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.281815 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.282039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.282206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.282317 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.282426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.282528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.282699 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.282893 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.283103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.283343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.283578 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.283809 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.284039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.284268 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.284511 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.284755 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.285032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.285531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.286584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] publish_errors = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.286584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.286584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.286584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.286984 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.287064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.287136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.287264 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.287345 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.287345 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.287519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] standard_threads = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.287601 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.287678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.287678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.287876 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.287950 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.288019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.288087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] use_journal = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.288156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.288232 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] use_json = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.288232 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] use_stderr = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.288370 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.288440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] use_syslog = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.288528 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.288679 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.288751 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.288820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.288889 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.288957 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.289094 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.289233 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.290836 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.290949 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.291027 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.291109 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] publish_errors = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.291181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.291256 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.291328 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.291404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.291485 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.291485 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.291625 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.291696 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.291765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.291834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.291911 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.291982 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.292052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] standard_threads = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.292162 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.292246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.292397 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.292482 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.292619 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.292695 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.292793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] use_journal = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.292913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.293031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] use_json = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.293168 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.293245 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] use_stderr = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.293367 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] use_syslog = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.293445 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.293598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 19:56:37.293675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.293854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.293933 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.294102 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.294183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.296599 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.296713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.296877 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.297025 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.297102 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.297172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.297242 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.297311 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.297404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.297477 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.297556 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.297628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.297737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.297820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.297820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.297958 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.298033 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.298140 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.298140 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.298269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.298269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.298413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.298483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.298557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.298628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.298704 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.298775 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.298844 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.298997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.299083 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.299155 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.299235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.299312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.299383 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.299451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.299525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.299600 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.299678 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.299752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.299827 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.299902 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.299973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.300075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.300177 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.300278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.300374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.300468 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.300568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.300662 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.300768 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.300896 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.301002 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.301099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.301213 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.301288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.301411 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.301493 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.301570 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.301649 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.301715 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.301784 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.301857 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.301928 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.301992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.302055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.302146 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.302218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.302303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.302376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.302450 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.302521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.302592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.302667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.302743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.302823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.302895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.302960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.303025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.303147 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.303226 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.303383 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.303458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.303607 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.303691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.303840 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.303919 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.304082 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.304156 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.304308 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.304392 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.304538 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.304621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.304764 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.304843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.304994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.305077 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.305310 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.305388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.305563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.305691 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.305806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.305962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.306039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.306250 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.306403 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.306508 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.306590 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.306763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.306876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.307019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.307106 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.307270 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.307354 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.307514 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.307591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.307776 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.307861 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.308008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.308085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.308241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.308317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.308473 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.308559 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.308705 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.308791 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.308960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.309035 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.309194 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.309283 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.309432 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.309517 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.309668 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.309743 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.309906 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.309988 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.310189 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.310286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.310441 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.310520 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.310685 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.310762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.310914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.310999 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.311145 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.311223 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.311375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.311458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.311609 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.311741 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.311831 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.311980 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.312058 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.312217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.312305 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.312457 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.312534 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.312693 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.312770 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.312930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.313019 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.313164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.313241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.313398 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.313474 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.313632 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.313717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] auth.external = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.313888 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.313970 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.314144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.314223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.314388 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.314478 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.314625 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.314702 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] auth.password = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.314852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] auth.token = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.314936 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.315089 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.315178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.315320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.315398 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.315563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.315642 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.315790 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.315877 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.316034 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.316110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.316263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.316341 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.316489 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.316574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.316712 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.316806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] credential.caching = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.316950 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.317029 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.317183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.317261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.317424 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.317509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.317654 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.317737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.317892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.317969 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.318143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.318236 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.318403 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.318481 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.318632 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] auth.external = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.318711 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.318859 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.318946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.319110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.319187 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.319365 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.319442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.319593 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] auth.password = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.319750 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.319825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] auth.token = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.319896 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.320043 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.320122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.320272 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.320358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.320512 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.320596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] federation.caching = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.320737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.320816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.320980 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.321066 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.321220 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.321298 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.321445 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.321531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.321674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] credential.caching = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.321762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.321914 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.321992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.322178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.322262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.322413 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.322500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.322641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.322718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.322879 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.322957 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.323108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.323194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.323339 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] identity.caching = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.323416 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.323590 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.323674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.323825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.323912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.324057 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.324141 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.324290 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.324368 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.324536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.324616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.324766 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.324843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.325002 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.325080 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.325251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.325331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.325490 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] federation.caching = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.325567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.325717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.325794 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.325948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.326030 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.326208 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.326294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.326436 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.326566 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.326679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.326821 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.326905 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.327069 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.327147 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.327326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.327413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.327575 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.327653 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.327813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.327889 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.328055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.328142 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.328284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] identity.caching = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.328362 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.328511 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.328594 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.328746 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.328831 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.328974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.329051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.329202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.329280 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.329438 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.329524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.329661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.329737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.329892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.329969 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.330147 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.330245 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.330396 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.330474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.330619 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.330696 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.330845 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.330931 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.331071 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.331149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.331311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.331390 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.331537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.331623 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.331772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.331851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.332000 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.332078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.332234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.332320 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.332469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.332547 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.332708 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.332785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.332943 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.333028 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.333177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.333257 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.333428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.333506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.333656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.333741 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.333889 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.333967 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.334138 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.334216 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.334376 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.334455 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.334619 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.334697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.334846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.334923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.335076 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.335154 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.335310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.335388 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.335539 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.335617 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.335777 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.335856 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.336048 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.336167 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.336277 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.336358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.337018 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.337106 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.337245 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.337323 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.337478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.337556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.337710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.337789 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.337943 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.338022 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.338197 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.338279 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.338431 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.338512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.338667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.338744 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.338897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.338975 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.339122 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.339201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.339348 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.339426 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.339592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.339670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.339832 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.339910 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.340060 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.340137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.340290 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.340368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.340520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.340617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.340746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.340823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.340977 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.341054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.341213 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.341292 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.341443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.341519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.341683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.341759 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.341922 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.342000 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.342177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.342258 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.342407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.342483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.342634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.342711 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.342853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.342932 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.343082 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.343160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.343313 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.343392 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.343537 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.343615 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.343778 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.343855 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.344016 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.344100 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.344244 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.344321 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.344478 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.344556 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.344715 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] resource.caching = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.344795 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.344940 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.345016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.345181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.345259 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.345415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.345494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.345644 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.345721 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.345875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.345960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.346119 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.346202 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.346373 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.346460 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.346607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.346685 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.346846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.346926 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.347078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] role.caching = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.347156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.347321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] role.driver = sql {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.347399 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.347550 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.347629 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.347781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.347858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.348011 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.348089 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.348244 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.348324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.348483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.348561 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.348719 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.348806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.348948 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.349027 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.349179 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.349258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.349412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] resource.caching = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.349499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.349639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.349716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.349884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.349962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.350130 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.350222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.350377 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.350455 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.350618 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.350695 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.350844 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.351055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.351136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.351291 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.351369 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.351524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.351600 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.351752 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] role.caching = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.351833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.351993 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] role.driver = sql {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.352072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.352785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.352898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.353107 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.353188 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.353353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.353432 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.353598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.353676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.353835 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.353914 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.354071 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.354172 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.354325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.354403 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.354558 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.354638 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.354807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.354884 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.355031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.355108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.355275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.355355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.355514 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.355590 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.355747 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.355825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.355974 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.356053 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.356206 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] token.caching = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.356396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.356474 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.356631 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.356710 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.356876 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.356954 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.357123 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.357202 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.357365 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.357446 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.357597 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.357674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.357826 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.357903 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.358066 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.358164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.358334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.358414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.358568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.358645 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.358806 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.358889 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.359009 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.359088 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.359239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.359317 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.359483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.359562 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.359711 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.359789 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.359951 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.360029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.360179 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.360257 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.360408 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.360486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.connection = **** {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.360632 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.360719 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.360862 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] token.caching = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.360948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.361091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.361169 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.361332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.361409 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.361562 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.361649 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.361823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.361908 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.362055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.362206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.362309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.362441 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.362550 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.362682 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.362783 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.362920 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.363023 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.363162 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.363242 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.363392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.363477 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.363626 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.363706 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.363852 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.363927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.364081 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.364160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.364322 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.364400 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.364545 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.364625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.364751 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80515) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 10 19:56:37.364829 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.365060 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.365315 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.connection = **** {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.365559 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.365802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.366035 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.366305 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.366535 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.366765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.366995 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.367225 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.367463 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.367694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.367941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.368170 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.368410 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.368640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.368879 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.369105 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.369341 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.369570 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 19:56:37.369737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80516) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 10 19:56:37.395781 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.396007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.396234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.397242 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.397472 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.397694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.405544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.406858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.434763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.434999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.435608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.435838 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.488941 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.489078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.541269 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.541463 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.541591 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.541702 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.607555 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.607777 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.607963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.608052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.608198 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.608280 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.608408 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.608489 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.608619 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.608701 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.608831 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.609045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.632225 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.632453 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.632546 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.632666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.632748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.632876 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.632968 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.633083 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.633166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.633288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.633988 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.634068 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.634174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.634309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.645935 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.646211 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.661324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.661643 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.661745 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.661828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.661903 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.661978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.662147 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.662235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.662372 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.662456 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.662590 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.662673 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80516) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.662805 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.663019 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80515) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 19:56:37.679561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.679762 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 10 19:56:37.679994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.680094 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.680220 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 10 19:56:37.680303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 10 19:56:37.680552 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.680632 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.680750 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 10 19:56:37.680831 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.681021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.681101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.681278 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.681357 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.681532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.681786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.692280 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.692551 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.692780 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.692874 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.693055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.693137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.693323 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.693604 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.701702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80516) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.702283 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80515) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.704991 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80516) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.705263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.705543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.705650 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80515) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.705825 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.705931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.706129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.706236 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.706495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.706586 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.706759 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.706839 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.707029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.707112 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.707294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.707382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.707568 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.707646 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.707824 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.707901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.708120 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.708385 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.735051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.735874 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.736671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80516) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.736951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.737514 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80515) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.737790 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.742335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80516) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.742568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.742865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.743195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80515) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.743471 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.743749 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.750502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80516) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.750774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.751418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80515) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.751697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.755399 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80516) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.755680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.755955 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.756233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.756347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80515) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.756500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.756633 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.756767 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.756914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.757029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.757199 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.757279 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.757474 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.757746 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.758016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.758310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.776935 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.777205 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.778033 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.778337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.781717 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80516) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.782865 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80515) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.785563 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.785851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.786507 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.786802 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.790890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80516) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.792327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80515) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.794480 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80516) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.795926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80515) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.798211 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.798503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.798782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.799061 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.799338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.799610 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.799692 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.799958 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.800240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.800523 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.800798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.801078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.818562 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80516) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.818933 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.819198 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.819458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.820166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80515) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.820542 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.821310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.821593 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.827865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.829519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.829603 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.829829 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.831283 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.831634 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.832907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80516) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.833181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.833458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.833741 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.834011 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.834698 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80515) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.834976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.835262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.835550 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.835824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.847995 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80516) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.848263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.848539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.848822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.849096 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.849364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.849641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.849865 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80515) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.850226 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.850448 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.850734 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.851013 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.851299 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.851898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.867723 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80516) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.869540 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80515) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.871296 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80516) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.873090 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80515) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.874611 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.876193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.876441 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.877814 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80516) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.878043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.878152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.878383 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.879718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80515) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.879992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.880279 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.886373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.887972 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80516) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.888246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.889870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80515) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.891210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.891494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.891777 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.892048 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.893127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.893415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.893697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.893972 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.901847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80516) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.902163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.902440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.903799 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80515) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.904091 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.904372 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.910315 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80516) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.910590 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.910864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.911143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.911409 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.911686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.911952 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.912220 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.912470 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80515) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 19:56:37.912550 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.912745 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.912834 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.913021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.913102 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.913294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.913374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.913554 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.913635 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.913811 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80516) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.913891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.914105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.914394 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.914689 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.914955 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.915221 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.915497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.915767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.916025 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80515) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 19:56:37.952801 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5582e998f9c0 pid: 80516 (default app) May 10 19:56:37.955093 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5582e998f9c0 pid: 80515 (default app) May 10 19:56:37.956128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b35f01-71a7-448e-94a1-8dcb5188a2c9 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:37.956423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b35f01-71a7-448e-94a1-8dcb5188a2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:37.956708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b35f01-71a7-448e-94a1-8dcb5188a2c9 None None] PATH_INFO: `/v3` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:37.960886 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1/1] 10.222.0.65 () {50 vars in 791 bytes} [Tue May 10 19:56:37 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:56:46.723415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7aa11-d5e6-490f-8757-2d1251564207 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:46.723988 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7aa11-d5e6-490f-8757-2d1251564207 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:46.723988 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7aa11-d5e6-490f-8757-2d1251564207 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:46.728247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1/2] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:56:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:56:46.733370 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d008ae-319c-40c8-979c-49ab9d27c53c None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:46.733777 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d008ae-319c-40c8-979c-49ab9d27c53c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:46.733916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d008ae-319c-40c8-979c-49ab9d27c53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:46.764792 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG oslo_db.sqlalchemy.engines [None req-02d008ae-319c-40c8-979c-49ab9d27c53c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80516) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 10 19:56:46.947640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG passlib.handlers.bcrypt [None req-02d008ae-319c-40c8-979c-49ab9d27c53c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80516) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 10 19:56:46.948057 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG passlib.handlers.bcrypt [None req-02d008ae-319c-40c8-979c-49ab9d27c53c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80516) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 10 19:56:47.003636 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-02d008ae-319c-40c8-979c-49ab9d27c53c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:56:47.005316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-02d008ae-319c-40c8-979c-49ab9d27c53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:56:47.127755 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 2/3] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:56:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 10 19:56:47.294932 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7aa11-d5e6-490f-8757-2d1251564207 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:47.329028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG oslo_db.sqlalchemy.engines [None req-caf7aa11-d5e6-490f-8757-2d1251564207 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80515) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 10 19:56:47.453650 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7aa11-d5e6-490f-8757-2d1251564207 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:47.457993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e10a66-9930-4202-8f2b-e60222ba7e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:47.458674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e10a66-9930-4202-8f2b-e60222ba7e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:47.458956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e10a66-9930-4202-8f2b-e60222ba7e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:47.459230 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e10a66-9930-4202-8f2b-e60222ba7e59 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:47.467235 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e10a66-9930-4202-8f2b-e60222ba7e59 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:47.519153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: WARNING keystone.common.rbac_enforcer.enforcer [None req-86e10a66-9930-4202-8f2b-e60222ba7e59 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 10 19:56:47.519488 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e10a66-9930-4202-8f2b-e60222ba7e59 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:47.528670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 2/4] 10.222.0.65 () {58 vars in 1324 bytes} [Tue May 10 19:56:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 19:56:47.534574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d008ae-319c-40c8-979c-49ab9d27c53c None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:47.549144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d008ae-319c-40c8-979c-49ab9d27c53c None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:47.553464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac08bf-b28c-4a66-ad68-8545859ff34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:47.554634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac08bf-b28c-4a66-ad68-8545859ff34b admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:47.555404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac08bf-b28c-4a66-ad68-8545859ff34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:47.555687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac08bf-b28c-4a66-ad68-8545859ff34b admin admin] PATH_INFO: `/v3/projects/475f061ace89472aa69923909a044007` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:47.557215 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac08bf-b28c-4a66-ad68-8545859ff34b admin admin] RBAC: Authorizing `identity:get_project(project_id=475f061ace89472aa69923909a044007)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:47.606917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: WARNING keystone.common.rbac_enforcer.enforcer [None req-50ac08bf-b28c-4a66-ad68-8545859ff34b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 10 19:56:47.607252 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac08bf-b28c-4a66-ad68-8545859ff34b admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:47.611707 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 3/5] 10.222.0.65 () {56 vars in 1207 bytes} [Tue May 10 19:56:47 2022] GET /identity/v3/projects/475f061ace89472aa69923909a044007 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:56:49.797353 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d7777-ad4b-42c2-a075-7068afd84791 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:49.797899 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d7777-ad4b-42c2-a075-7068afd84791 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:49.797899 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d7777-ad4b-42c2-a075-7068afd84791 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:49.802048 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 3/6] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:56:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:56:49.808238 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb7ee4-ef92-4b97-8258-45b16a29994e None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:49.808617 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb7ee4-ef92-4b97-8258-45b16a29994e None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:49.808787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb7ee4-ef92-4b97-8258-45b16a29994e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:49.828553 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-3deb7ee4-ef92-4b97-8258-45b16a29994e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:56:49.830067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-3deb7ee4-ef92-4b97-8258-45b16a29994e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:56:49.837926 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 4/7] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:56:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 10 19:56:49.859878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d7777-ad4b-42c2-a075-7068afd84791 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:49.872498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d7777-ad4b-42c2-a075-7068afd84791 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:49.876678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693641e-542c-4432-8685-54de22e24505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:49.877313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693641e-542c-4432-8685-54de22e24505 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:49.877598 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693641e-542c-4432-8685-54de22e24505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:49.877904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693641e-542c-4432-8685-54de22e24505 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:49.885447 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693641e-542c-4432-8685-54de22e24505 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:49.887240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693641e-542c-4432-8685-54de22e24505 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:49.895823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 4/8] 10.222.0.65 () {58 vars in 1324 bytes} [Tue May 10 19:56:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 19:56:49.900915 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb7ee4-ef92-4b97-8258-45b16a29994e None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:49.909630 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb7ee4-ef92-4b97-8258-45b16a29994e None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:49.914519 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12a6fe-b835-4dfc-bde3-bdeead08cdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:49.915370 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12a6fe-b835-4dfc-bde3-bdeead08cdcf admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:49.915651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12a6fe-b835-4dfc-bde3-bdeead08cdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:49.915922 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12a6fe-b835-4dfc-bde3-bdeead08cdcf admin admin] PATH_INFO: `/v3/users/729fa49ea5f943b2b09c35b636e34ee5` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:49.917337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12a6fe-b835-4dfc-bde3-bdeead08cdcf admin admin] RBAC: Authorizing `identity:get_user(user_id=729fa49ea5f943b2b09c35b636e34ee5)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:49.919173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12a6fe-b835-4dfc-bde3-bdeead08cdcf admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:49.920716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 5/9] 10.222.0.65 () {56 vars in 1198 bytes} [Tue May 10 19:56:49 2022] GET /identity/v3/users/729fa49ea5f943b2b09c35b636e34ee5 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:56:52.165086 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a15ef3-78ce-42c8-a415-145a4bbeb0f9 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:52.165626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a15ef3-78ce-42c8-a415-145a4bbeb0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:52.165626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a15ef3-78ce-42c8-a415-145a4bbeb0f9 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:52.167002 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 5/10] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:56:52.172089 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c08197-41c9-4ff0-a396-62a1d38f9039 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:52.172471 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c08197-41c9-4ff0-a396-62a1d38f9039 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:52.172657 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c08197-41c9-4ff0-a396-62a1d38f9039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:52.238041 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f8209-ff89-4936-82ed-4181b5241a05 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:52.238384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f8209-ff89-4936-82ed-4181b5241a05 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:52.238666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f8209-ff89-4936-82ed-4181b5241a05 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:52.240031 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 6/11] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:56:52.246847 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad077f-c880-4a65-bb21-88e8ec7d71db None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:52.247137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad077f-c880-4a65-bb21-88e8ec7d71db None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:52.247414 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad077f-c880-4a65-bb21-88e8ec7d71db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:52.288159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG passlib.handlers.bcrypt [None req-2dad077f-c880-4a65-bb21-88e8ec7d71db None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80515) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 10 19:56:52.288596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG passlib.handlers.bcrypt [None req-2dad077f-c880-4a65-bb21-88e8ec7d71db None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80515) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 10 19:56:52.296852 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-99c08197-41c9-4ff0-a396-62a1d38f9039 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:56:52.298055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-99c08197-41c9-4ff0-a396-62a1d38f9039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:56:52.335542 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-2dad077f-c880-4a65-bb21-88e8ec7d71db None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:56:52.337095 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-2dad077f-c880-4a65-bb21-88e8ec7d71db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:56:52.386637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 6/12] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:56:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:56:52.409095 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c08197-41c9-4ff0-a396-62a1d38f9039 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:52.436587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 7/13] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:56:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:56:52.459036 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad077f-c880-4a65-bb21-88e8ec7d71db None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:52.505140 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c08197-41c9-4ff0-a396-62a1d38f9039 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:52.537126 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ec888-bd25-456f-be3f-0f3537ded8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:52.537911 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ec888-bd25-456f-be3f-0f3537ded8cc None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:52.538223 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ec888-bd25-456f-be3f-0f3537ded8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:52.538502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ec888-bd25-456f-be3f-0f3537ded8cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:52.549220 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ec888-bd25-456f-be3f-0f3537ded8cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:56:52.553870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad077f-c880-4a65-bb21-88e8ec7d71db None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:52.560802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-487ec888-bd25-456f-be3f-0f3537ded8cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:56:52.560802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:56:52.560802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:56:52.560802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:56:52.560802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:56:52.560802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:56:52.560802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:56:52.560802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:56:52.560802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:56:52.560802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:56:52.560802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:56:52.560802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:56:52.560802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:56:52.560802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:56:52.560802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:56:52.560802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:56:52.561842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:56:52.561842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:56:52.561842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:56:52.561842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:56:52.561842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:56:52.561842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:56:52.561842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:56:52.561842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:56:52.561842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:56:52.561842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:56:52.561842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:56:52.561842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:56:52.561842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:56:52.561842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:56:52.561842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:56:52.561842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:56:52.562727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:56:52.562727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:56:52.562727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:56:52.562727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:56:52.562727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:56:52.562727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:56:52.562727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:56:52.562727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:56:52.562727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:56:52.562727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:56:52.562727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:56:52.562727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:56:52.562727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 7/14] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:56:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:56:52.567692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ec888-bd25-456f-be3f-0f3537ded8cc None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:52.587845 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724925a-1f4e-4d9c-a56f-23c22fcb99f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:52.588751 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724925a-1f4e-4d9c-a56f-23c22fcb99f2 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:52.589038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724925a-1f4e-4d9c-a56f-23c22fcb99f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:52.589320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724925a-1f4e-4d9c-a56f-23c22fcb99f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:52.653236 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724925a-1f4e-4d9c-a56f-23c22fcb99f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:52.654779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724925a-1f4e-4d9c-a56f-23c22fcb99f2 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:52.663240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ec888-bd25-456f-be3f-0f3537ded8cc None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:52.699459 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c20ad6-acc8-46bb-a495-7cf47717d7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:52.700119 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c20ad6-acc8-46bb-a495-7cf47717d7bf None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:52.700405 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c20ad6-acc8-46bb-a495-7cf47717d7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:52.700677 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c20ad6-acc8-46bb-a495-7cf47717d7bf None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:52.701432 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c20ad6-acc8-46bb-a495-7cf47717d7bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:56:52.701734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c20ad6-acc8-46bb-a495-7cf47717d7bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:52.703233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c20ad6-acc8-46bb-a495-7cf47717d7bf None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:52.715028 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 8/15] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:56:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:56:52.720525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c20ad6-acc8-46bb-a495-7cf47717d7bf None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:52.754587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 8/16] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:56:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 19:56:52.760543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724925a-1f4e-4d9c-a56f-23c22fcb99f2 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:52.814427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c20ad6-acc8-46bb-a495-7cf47717d7bf None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:52.848128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18666aae-b269-4c68-8812-73d7619c078c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:52.848983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18666aae-b269-4c68-8812-73d7619c078c None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:52.849260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18666aae-b269-4c68-8812-73d7619c078c None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:52.849529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18666aae-b269-4c68-8812-73d7619c078c None admin] PATH_INFO: `/v3/users/729fa49ea5f943b2b09c35b636e34ee5` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:52.850744 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18666aae-b269-4c68-8812-73d7619c078c None admin] RBAC: Authorizing `identity:get_user(user_id=729fa49ea5f943b2b09c35b636e34ee5)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:52.852211 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18666aae-b269-4c68-8812-73d7619c078c None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:52.853295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724925a-1f4e-4d9c-a56f-23c22fcb99f2 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:52.853807 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 9/17] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:56:52 2022] GET /identity/v3/users/729fa49ea5f943b2b09c35b636e34ee5 => generated 247 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:56:52.859022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18666aae-b269-4c68-8812-73d7619c078c None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:52.885923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e00591-1fb6-480d-a273-49c728f2be14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:52.886746 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e00591-1fb6-480d-a273-49c728f2be14 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:52.887031 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e00591-1fb6-480d-a273-49c728f2be14 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:52.887307 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e00591-1fb6-480d-a273-49c728f2be14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:52.957360 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18666aae-b269-4c68-8812-73d7619c078c None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:52.992650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d821b-cc58-4864-969b-eb1f1c5ee4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:52.993446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d821b-cc58-4864-969b-eb1f1c5ee4fb None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:52.993732 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d821b-cc58-4864-969b-eb1f1c5ee4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:52.994002 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d821b-cc58-4864-969b-eb1f1c5ee4fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:52.995479 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1d821b-cc58-4864-969b-eb1f1c5ee4fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:52.996924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1d821b-cc58-4864-969b-eb1f1c5ee4fb None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:52.998569 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 10/18] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:56:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:56:53.004350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d821b-cc58-4864-969b-eb1f1c5ee4fb None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:53.013797 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e00591-1fb6-480d-a273-49c728f2be14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:53.015302 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e00591-1fb6-480d-a273-49c728f2be14 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:53.036495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-68e00591-1fb6-480d-a273-49c728f2be14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:56:53.036495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:56:53.036495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:56:53.036495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:56:53.036495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:56:53.036495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:56:53.036495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:56:53.036495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:56:53.036495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:56:53.036495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:56:53.036495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:56:53.036495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:56:53.036495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:56:53.036495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:56:53.036495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:56:53.037519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:56:53.037519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:56:53.037519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:56:53.037519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:56:53.037519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:56:53.037519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:56:53.037519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:56:53.037519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:56:53.037519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:56:53.037519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:56:53.037519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:56:53.037519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:56:53.037519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:56:53.037519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:56:53.037519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:56:53.037519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:56:53.037519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:56:53.038583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:56:53.038583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:56:53.038583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:56:53.038583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:56:53.038583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 9/19] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:56:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:56:53.044273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e00591-1fb6-480d-a273-49c728f2be14 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:53.103646 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d821b-cc58-4864-969b-eb1f1c5ee4fb None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:53.135391 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771f735-289f-4e5e-a91e-7028cf1346d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:53.136030 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771f735-289f-4e5e-a91e-7028cf1346d9 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:53.136311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771f735-289f-4e5e-a91e-7028cf1346d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:53.136582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771f735-289f-4e5e-a91e-7028cf1346d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:53.137417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771f735-289f-4e5e-a91e-7028cf1346d9 None admin] RBAC: Adding query filter params (role.id=5c87e804d0764309ba08ad6e06b23a87, scope.domain.id=default, user.id=729fa49ea5f943b2b09c35b636e34ee5) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:56:53.137723 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771f735-289f-4e5e-a91e-7028cf1346d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:53.139169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771f735-289f-4e5e-a91e-7028cf1346d9 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:53.142898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e00591-1fb6-480d-a273-49c728f2be14 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:53.145773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 11/20] 10.222.0.65 () {56 vars in 1322 bytes} [Tue May 10 19:56:53 2022] GET /identity/v3/role_assignments?user.id=729fa49ea5f943b2b09c35b636e34ee5&scope.domain.id=default&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 233 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:56:53.178204 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee360e-5b5e-4bd8-9863-180f96911277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:53.178884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee360e-5b5e-4bd8-9863-180f96911277 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:53.179239 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee360e-5b5e-4bd8-9863-180f96911277 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:53.179516 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee360e-5b5e-4bd8-9863-180f96911277 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:53.180274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee360e-5b5e-4bd8-9863-180f96911277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:56:53.180584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee360e-5b5e-4bd8-9863-180f96911277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:53.182000 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee360e-5b5e-4bd8-9863-180f96911277 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:53.200986 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 10/21] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:56:55.633593 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991d3d9-50aa-4487-8661-fa0ed6cd1991 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:55.634147 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991d3d9-50aa-4487-8661-fa0ed6cd1991 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:55.634224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991d3d9-50aa-4487-8661-fa0ed6cd1991 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:55.635551 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 12/22] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:56:55.641171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7afa43-c1cc-4ec6-a50a-30a528f5fd35 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:55.642009 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7afa43-c1cc-4ec6-a50a-30a528f5fd35 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:55.642507 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7afa43-c1cc-4ec6-a50a-30a528f5fd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:55.665503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-3c7afa43-c1cc-4ec6-a50a-30a528f5fd35 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:56:55.667085 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3c7afa43-c1cc-4ec6-a50a-30a528f5fd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:56:55.752478 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 11/23] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:56:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:56:55.793803 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991d3d9-50aa-4487-8661-fa0ed6cd1991 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:55.930783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991d3d9-50aa-4487-8661-fa0ed6cd1991 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:55.969109 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aba1ac-dc73-49b2-b438-745957155989 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:55.969454 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aba1ac-dc73-49b2-b438-745957155989 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:55.969677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aba1ac-dc73-49b2-b438-745957155989 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:55.971264 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 12/24] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:56:55.982909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d58ba-886d-423b-968b-383a71a0e1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:55.983727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d58ba-886d-423b-968b-383a71a0e1ec None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:55.984011 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d58ba-886d-423b-968b-383a71a0e1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:55.984288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d58ba-886d-423b-968b-383a71a0e1ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:55.989157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792741d-6581-4117-8305-b06ed0724531 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:55.989157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792741d-6581-4117-8305-b06ed0724531 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:55.989157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792741d-6581-4117-8305-b06ed0724531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:56.004359 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d58ba-886d-423b-968b-383a71a0e1ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:56:56.019133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-e792741d-6581-4117-8305-b06ed0724531 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:56:56.020339 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-e792741d-6581-4117-8305-b06ed0724531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:56:56.020446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-c25d58ba-886d-423b-968b-383a71a0e1ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:56:56.020446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:56:56.020446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:56:56.020446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:56:56.020446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:56:56.020446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:56:56.020446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:56:56.020446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:56:56.020446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:56:56.020446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:56:56.020446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:56:56.020446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:56:56.020446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:56:56.020446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:56:56.020446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:56:56.020446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:56:56.021232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:56:56.021232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:56:56.021232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:56:56.021232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:56:56.021232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:56:56.021232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:56:56.021232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:56:56.021232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:56:56.021232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:56:56.021232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:56:56.021232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:56:56.021232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:56:56.021232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:56:56.021232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:56:56.021232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:56:56.021232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:56:56.022449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:56:56.022449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:56:56.022449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:56:56.022449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:56:56.022449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:56:56.022449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:56:56.022449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:56:56.022449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:56:56.022449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:56:56.022449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:56:56.022449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:56:56.022449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:56:56.022449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 13/25] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:56:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:56:56.028996 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d58ba-886d-423b-968b-383a71a0e1ec None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:56.149135 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 13/26] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:56:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:56:56.182747 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fd4c9-4a01-4bba-a336-ce6b379b52cb None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:56.183072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fd4c9-4a01-4bba-a336-ce6b379b52cb None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:56.183324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fd4c9-4a01-4bba-a336-ce6b379b52cb None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:56.184950 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 14/27] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:56:56.193583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fd4c9-4a01-4bba-a336-ce6b379b52cb None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:56.225009 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d58ba-886d-423b-968b-383a71a0e1ec None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:56.281536 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c0bd6-415b-492e-a633-d72404fa4213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:56.282152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c0bd6-415b-492e-a633-d72404fa4213 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:56.282442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c0bd6-415b-492e-a633-d72404fa4213 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:56.282714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c0bd6-415b-492e-a633-d72404fa4213 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:56.283477 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022c0bd6-415b-492e-a633-d72404fa4213 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:56:56.283782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022c0bd6-415b-492e-a633-d72404fa4213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:56.290526 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022c0bd6-415b-492e-a633-d72404fa4213 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:56.304679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 14/28] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:56:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:56:56.306199 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56157a2-c0ca-4b2b-913e-77d74b8f0ce0 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:56.306703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56157a2-c0ca-4b2b-913e-77d74b8f0ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:56.307175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56157a2-c0ca-4b2b-913e-77d74b8f0ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:56.324389 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-e56157a2-c0ca-4b2b-913e-77d74b8f0ce0 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:56:56.325890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-e56157a2-c0ca-4b2b-913e-77d74b8f0ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:56:56.344082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fd4c9-4a01-4bba-a336-ce6b379b52cb None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:56.416927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ad99b-4efc-4783-9a1a-a326715dd6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:56.418237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ad99b-4efc-4783-9a1a-a326715dd6cb None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:56.418739 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ad99b-4efc-4783-9a1a-a326715dd6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:56.419208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ad99b-4efc-4783-9a1a-a326715dd6cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:56.438647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ad99b-4efc-4783-9a1a-a326715dd6cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:56.440339 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ad99b-4efc-4783-9a1a-a326715dd6cb None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:56.452610 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 15/29] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:56:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:56:56.453247 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56157a2-c0ca-4b2b-913e-77d74b8f0ce0 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:56.459946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-dc9ad99b-4efc-4783-9a1a-a326715dd6cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:56:56.459946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:56:56.459946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:56:56.459946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:56:56.459946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:56:56.459946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:56:56.459946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:56:56.459946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:56:56.459946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:56:56.459946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:56:56.459946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:56:56.459946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:56:56.459946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:56:56.459946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:56:56.459946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:56:56.461343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:56:56.461343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:56:56.461343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:56:56.461343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:56:56.461343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:56:56.461343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:56:56.461343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:56:56.461343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:56:56.461343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:56:56.461343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:56:56.461343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:56:56.461343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:56:56.461343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:56:56.461343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:56:56.461343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:56:56.461343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:56:56.461343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:56:56.462184 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:56:56.462184 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:56:56.462184 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:56:56.462184 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:56:56.464157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 15/30] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:56:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:56:56.475095 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ad99b-4efc-4783-9a1a-a326715dd6cb None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:56.618453 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56157a2-c0ca-4b2b-913e-77d74b8f0ce0 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:56.624527 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ad99b-4efc-4783-9a1a-a326715dd6cb None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:56.658351 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71010df2-306c-42a8-9f15-71a877056213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:56.659142 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71010df2-306c-42a8-9f15-71a877056213 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:56.659496 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71010df2-306c-42a8-9f15-71a877056213 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:56.659843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71010df2-306c-42a8-9f15-71a877056213 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:56.660649 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71010df2-306c-42a8-9f15-71a877056213 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:56:56.661024 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71010df2-306c-42a8-9f15-71a877056213 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:56.661132 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b39799-8bd6-4f21-a708-072921085f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:56.662012 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b39799-8bd6-4f21-a708-072921085f92 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:56.662335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b39799-8bd6-4f21-a708-072921085f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:56.662615 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b39799-8bd6-4f21-a708-072921085f92 None admin] PATH_INFO: `/v3/users/729fa49ea5f943b2b09c35b636e34ee5` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:56.662736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71010df2-306c-42a8-9f15-71a877056213 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:56.663861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b39799-8bd6-4f21-a708-072921085f92 None admin] RBAC: Authorizing `identity:get_user(user_id=729fa49ea5f943b2b09c35b636e34ee5)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:56.665957 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b39799-8bd6-4f21-a708-072921085f92 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:56.667971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 16/31] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:56:56 2022] GET /identity/v3/users/729fa49ea5f943b2b09c35b636e34ee5 => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:56:56.668823 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b39799-8bd6-4f21-a708-072921085f92 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:56.678349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 16/32] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:56:56.679820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4d5f5-16c4-4726-9ead-e404892bb21c None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:56.680117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4d5f5-16c4-4726-9ead-e404892bb21c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:56.680398 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4d5f5-16c4-4726-9ead-e404892bb21c None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:56.681921 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 17/33] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:56:56.682657 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4d5f5-16c4-4726-9ead-e404892bb21c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:56.779399 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b39799-8bd6-4f21-a708-072921085f92 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:56.813497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e75543-af2b-46f2-8e2e-5d48d3a34be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:56.814204 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e75543-af2b-46f2-8e2e-5d48d3a34be5 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:56.814491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e75543-af2b-46f2-8e2e-5d48d3a34be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:56.814763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e75543-af2b-46f2-8e2e-5d48d3a34be5 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:56.815324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4d5f5-16c4-4726-9ead-e404892bb21c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:56.815499 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e75543-af2b-46f2-8e2e-5d48d3a34be5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:56.816880 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e75543-af2b-46f2-8e2e-5d48d3a34be5 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:56.860038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564d6ad-d6cf-4523-bd06-b79dab5b5bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:56.860858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564d6ad-d6cf-4523-bd06-b79dab5b5bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:56.861148 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564d6ad-d6cf-4523-bd06-b79dab5b5bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:56.861423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564d6ad-d6cf-4523-bd06-b79dab5b5bb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:56.862872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564d6ad-d6cf-4523-bd06-b79dab5b5bb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:56.864324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564d6ad-d6cf-4523-bd06-b79dab5b5bb3 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:56.865993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 18/34] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:56:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:56:56.867304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564d6ad-d6cf-4523-bd06-b79dab5b5bb3 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:56.970255 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564d6ad-d6cf-4523-bd06-b79dab5b5bb3 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:57.004527 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7ee52-0749-4d98-8933-2abe255bb28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:57.005421 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7ee52-0749-4d98-8933-2abe255bb28a None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:57.005877 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7ee52-0749-4d98-8933-2abe255bb28a None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:57.006320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7ee52-0749-4d98-8933-2abe255bb28a None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:57.007745 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde7ee52-0749-4d98-8933-2abe255bb28a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:57.009342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde7ee52-0749-4d98-8933-2abe255bb28a None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:57.050043 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 17/35] 10.222.0.65 () {60 vars in 1131 bytes} [Tue May 10 19:56:56 2022] POST /identity/v3/roles => generated 225 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:56:57.058135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc97b0-755b-4cae-9e87-d0c1dea44715 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:57.058135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc97b0-755b-4cae-9e87-d0c1dea44715 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:57.058135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc97b0-755b-4cae-9e87-d0c1dea44715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:57.072042 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-9fcc97b0-755b-4cae-9e87-d0c1dea44715 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:56:57.073661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-9fcc97b0-755b-4cae-9e87-d0c1dea44715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:56:57.171399 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 18/36] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:56:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:56:57.172261 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc97b0-755b-4cae-9e87-d0c1dea44715 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:57.296093 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 19/37] 10.222.0.65 () {60 vars in 1141 bytes} [Tue May 10 19:56:56 2022] POST /identity/v3/projects => generated 301 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:56:57.297544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7ee52-0749-4d98-8933-2abe255bb28a None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:57.307474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc97b0-755b-4cae-9e87-d0c1dea44715 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:57.347292 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19fcca-cd56-4ef7-b9c6-0cbbc59ec6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:57.348185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19fcca-cd56-4ef7-b9c6-0cbbc59ec6fc None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:57.348532 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19fcca-cd56-4ef7-b9c6-0cbbc59ec6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:57.348824 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19fcca-cd56-4ef7-b9c6-0cbbc59ec6fc None admin] PATH_INFO: `/v3/domains/default/users/729fa49ea5f943b2b09c35b636e34ee5/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:57.359784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19fcca-cd56-4ef7-b9c6-0cbbc59ec6fc None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=729fa49ea5f943b2b09c35b636e34ee5, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:57.363037 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19fcca-cd56-4ef7-b9c6-0cbbc59ec6fc None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:57.457136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 19/38] 10.222.0.65 () {58 vars in 1361 bytes} [Tue May 10 19:56:57 2022] PUT /identity/v3/domains/default/users/729fa49ea5f943b2b09c35b636e34ee5/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:56:57.459057 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8409dd-ed28-4907-851a-122818536e41 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:57.459345 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8409dd-ed28-4907-851a-122818536e41 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:57.459635 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8409dd-ed28-4907-851a-122818536e41 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:57.461149 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 20/39] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:56:57.467226 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383d9cf-64f5-4399-90d8-1b431b2e90a7 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:57.467543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383d9cf-64f5-4399-90d8-1b431b2e90a7 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:57.467832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383d9cf-64f5-4399-90d8-1b431b2e90a7 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:57.474998 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 21/40] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:56:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:56:57.477985 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee8a3f-782f-4b71-b0ea-0ebe946f7f3b None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:57.478907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee8a3f-782f-4b71-b0ea-0ebe946f7f3b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:57.479692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee8a3f-782f-4b71-b0ea-0ebe946f7f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:57.488595 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7ee52-0749-4d98-8933-2abe255bb28a None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:57.503185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-dfee8a3f-782f-4b71-b0ea-0ebe946f7f3b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:56:57.504815 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-dfee8a3f-782f-4b71-b0ea-0ebe946f7f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:56:57.541184 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf93534-e600-442a-a22c-44410f099796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:57.541906 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf93534-e600-442a-a22c-44410f099796 None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:57.542430 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf93534-e600-442a-a22c-44410f099796 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:57.542725 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf93534-e600-442a-a22c-44410f099796 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:57.544187 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf93534-e600-442a-a22c-44410f099796 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:57.546070 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf93534-e600-442a-a22c-44410f099796 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:57.585888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 22/41] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:56:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:56:57.587584 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dff9c-6258-4fac-8271-fbe3a8a6f620 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:57.587892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dff9c-6258-4fac-8271-fbe3a8a6f620 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:57.588161 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dff9c-6258-4fac-8271-fbe3a8a6f620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:57.606548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-c76dff9c-6258-4fac-8271-fbe3a8a6f620 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:56:57.607764 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c76dff9c-6258-4fac-8271-fbe3a8a6f620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:56:57.620491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 20/42] 10.222.0.65 () {60 vars in 1131 bytes} [Tue May 10 19:56:57 2022] POST /identity/v3/roles => generated 219 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:56:57.621712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf93534-e600-442a-a22c-44410f099796 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:57.688482 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 23/43] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:56:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:56:57.712746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dff9c-6258-4fac-8271-fbe3a8a6f620 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:57.737057 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf93534-e600-442a-a22c-44410f099796 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:57.781194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb470aec-6264-430b-9d94-48e8dde0cd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:57.782015 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb470aec-6264-430b-9d94-48e8dde0cd65 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:57.782336 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb470aec-6264-430b-9d94-48e8dde0cd65 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:57.782620 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb470aec-6264-430b-9d94-48e8dde0cd65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:57.784007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb470aec-6264-430b-9d94-48e8dde0cd65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:57.785503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb470aec-6264-430b-9d94-48e8dde0cd65 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:57.787265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 21/44] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:56:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:56:57.793977 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb470aec-6264-430b-9d94-48e8dde0cd65 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:57.831181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dff9c-6258-4fac-8271-fbe3a8a6f620 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:57.883281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d201970-4990-407c-8d81-a0a75f237015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:57.884760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d201970-4990-407c-8d81-a0a75f237015 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:57.884760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d201970-4990-407c-8d81-a0a75f237015 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:57.884760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d201970-4990-407c-8d81-a0a75f237015 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:57.885247 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d201970-4990-407c-8d81-a0a75f237015 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:57.886678 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d201970-4990-407c-8d81-a0a75f237015 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:57.919115 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb470aec-6264-430b-9d94-48e8dde0cd65 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:56:57.951888 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7f70b-5923-4429-9e20-781c8ba53c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:56:57.952558 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7f70b-5923-4429-9e20-781c8ba53c4b None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:57.952845 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7f70b-5923-4429-9e20-781c8ba53c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:57.953136 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7f70b-5923-4429-9e20-781c8ba53c4b None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:57.953874 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a7f70b-5923-4429-9e20-781c8ba53c4b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:56:57.955348 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a7f70b-5923-4429-9e20-781c8ba53c4b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:56:57.957927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 24/45] 10.222.0.65 () {60 vars in 1131 bytes} [Tue May 10 19:56:57 2022] POST /identity/v3/roles => generated 223 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 19:56:58.051544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 22/46] 10.222.0.65 () {60 vars in 1141 bytes} [Tue May 10 19:56:57 2022] POST /identity/v3/projects => generated 312 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:56:59.765185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a6a52-164b-4223-8d29-d6fe078ef477 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:59.765798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a6a52-164b-4223-8d29-d6fe078ef477 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:59.765798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a6a52-164b-4223-8d29-d6fe078ef477 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:59.767142 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 25/47] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:56:59.773118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38924a76-1509-4398-b41f-a8a7578b976c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:56:59.773540 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38924a76-1509-4398-b41f-a8a7578b976c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:56:59.773687 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38924a76-1509-4398-b41f-a8a7578b976c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:56:59.790715 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-38924a76-1509-4398-b41f-a8a7578b976c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:56:59.792231 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-38924a76-1509-4398-b41f-a8a7578b976c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:56:59.856599 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 23/48] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:56:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:56:59.879286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a6a52-164b-4223-8d29-d6fe078ef477 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:56:59.975497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a6a52-164b-4223-8d29-d6fe078ef477 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:00.007737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbe553-ce2f-4ac9-a795-9c96e0f999c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:00.008520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbe553-ce2f-4ac9-a795-9c96e0f999c2 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:00.008811 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbe553-ce2f-4ac9-a795-9c96e0f999c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:00.009093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbe553-ce2f-4ac9-a795-9c96e0f999c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:00.018858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbe553-ce2f-4ac9-a795-9c96e0f999c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:00.026944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-bbfbe553-ce2f-4ac9-a795-9c96e0f999c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:00.026944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:00.026944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:00.026944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:00.026944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:00.026944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:00.026944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:00.026944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:00.026944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:00.026944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:00.026944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:00.026944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:00.026944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:00.026944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:00.026944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:00.026944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:00.027785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:00.027785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:00.027785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:00.027785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:00.027785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:00.027785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:00.027785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:00.027785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:00.027785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:00.027785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:00.027785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:00.027785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:00.027785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:00.027785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:00.027785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:00.027785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:00.028539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:00.028539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:00.028539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:00.028539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:00.028539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:00.028539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:00.028539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:00.028539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:00.028539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:00.028539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:00.028539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:00.028539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:00.028539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 26/49] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:56:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:00.033833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38924a76-1509-4398-b41f-a8a7578b976c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:00.133343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38924a76-1509-4398-b41f-a8a7578b976c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:00.166557 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557ec60-476b-4126-810b-753c2eb4d2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:00.167231 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557ec60-476b-4126-810b-753c2eb4d2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:00.167521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557ec60-476b-4126-810b-753c2eb4d2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:00.167804 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557ec60-476b-4126-810b-753c2eb4d2e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:00.168570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557ec60-476b-4126-810b-753c2eb4d2e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:00.168882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557ec60-476b-4126-810b-753c2eb4d2e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:00.170297 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557ec60-476b-4126-810b-753c2eb4d2e4 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:00.181579 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 24/50] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:00.187099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbe553-ce2f-4ac9-a795-9c96e0f999c2 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:00.260875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656049af-c8ec-4933-8a2b-73820c73c187 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:00.261164 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656049af-c8ec-4933-8a2b-73820c73c187 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:00.261448 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656049af-c8ec-4933-8a2b-73820c73c187 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:00.262860 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 25/51] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:57:00.267946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a50465-95dd-48f6-9cf6-dcef58961cc2 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:00.268239 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a50465-95dd-48f6-9cf6-dcef58961cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:00.268523 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a50465-95dd-48f6-9cf6-dcef58961cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:00.285209 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-d0a50465-95dd-48f6-9cf6-dcef58961cc2 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:00.286378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-d0a50465-95dd-48f6-9cf6-dcef58961cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:00.286740 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbe553-ce2f-4ac9-a795-9c96e0f999c2 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:00.319208 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50091cc2-efd9-48f1-86e8-71d4f4c69bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:00.320604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50091cc2-efd9-48f1-86e8-71d4f4c69bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:00.320889 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50091cc2-efd9-48f1-86e8-71d4f4c69bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:00.321169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50091cc2-efd9-48f1-86e8-71d4f4c69bb5 None admin] PATH_INFO: `/v3/users/729fa49ea5f943b2b09c35b636e34ee5` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:00.322395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50091cc2-efd9-48f1-86e8-71d4f4c69bb5 None admin] RBAC: Authorizing `identity:get_user(user_id=729fa49ea5f943b2b09c35b636e34ee5)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:00.323851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50091cc2-efd9-48f1-86e8-71d4f4c69bb5 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:00.325518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 27/52] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:00 2022] GET /identity/v3/users/729fa49ea5f943b2b09c35b636e34ee5 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:00.330918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50091cc2-efd9-48f1-86e8-71d4f4c69bb5 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:00.350006 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 26/53] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:57:00.372579 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a50465-95dd-48f6-9cf6-dcef58961cc2 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:00.426250 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50091cc2-efd9-48f1-86e8-71d4f4c69bb5 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:00.458741 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68e164-5805-49e5-9bdb-8dc8ba447532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:00.459613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68e164-5805-49e5-9bdb-8dc8ba447532 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:00.459899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68e164-5805-49e5-9bdb-8dc8ba447532 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:00.460876 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68e164-5805-49e5-9bdb-8dc8ba447532 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:00.462056 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68e164-5805-49e5-9bdb-8dc8ba447532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:00.463530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68e164-5805-49e5-9bdb-8dc8ba447532 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:00.465119 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 28/54] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:57:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:00.468733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a50465-95dd-48f6-9cf6-dcef58961cc2 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:00.470802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68e164-5805-49e5-9bdb-8dc8ba447532 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:00.501918 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebb98a-a445-4aac-bfb0-c8c4eb4025e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:00.502800 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebb98a-a445-4aac-bfb0-c8c4eb4025e3 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:00.503091 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebb98a-a445-4aac-bfb0-c8c4eb4025e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:00.503376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebb98a-a445-4aac-bfb0-c8c4eb4025e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:00.504583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ebb98a-a445-4aac-bfb0-c8c4eb4025e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:00.506024 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ebb98a-a445-4aac-bfb0-c8c4eb4025e3 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:00.508251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 27/55] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:57:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:00.513796 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebb98a-a445-4aac-bfb0-c8c4eb4025e3 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:00.564948 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68e164-5805-49e5-9bdb-8dc8ba447532 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:00.598349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d298a-8743-4a9f-8a14-b1dc84206dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:00.599003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d298a-8743-4a9f-8a14-b1dc84206dae None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:00.599290 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d298a-8743-4a9f-8a14-b1dc84206dae None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:00.599575 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d298a-8743-4a9f-8a14-b1dc84206dae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:00.600408 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d298a-8743-4a9f-8a14-b1dc84206dae None admin] RBAC: Adding query filter params (role.id=5c87e804d0764309ba08ad6e06b23a87, scope.domain.id=default, user.id=729fa49ea5f943b2b09c35b636e34ee5) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:00.600750 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d298a-8743-4a9f-8a14-b1dc84206dae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:00.602198 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d298a-8743-4a9f-8a14-b1dc84206dae None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:00.607751 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebb98a-a445-4aac-bfb0-c8c4eb4025e3 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:00.607870 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 29/56] 10.222.0.65 () {56 vars in 1322 bytes} [Tue May 10 19:57:00 2022] GET /identity/v3/role_assignments?user.id=729fa49ea5f943b2b09c35b636e34ee5&scope.domain.id=default&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 530 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:00.643900 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d2a2b-66f5-4351-b65a-31c50618208b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:00.644581 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d2a2b-66f5-4351-b65a-31c50618208b None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:00.644872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d2a2b-66f5-4351-b65a-31c50618208b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:00.645155 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d2a2b-66f5-4351-b65a-31c50618208b None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:00.645893 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d2a2b-66f5-4351-b65a-31c50618208b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:00.647435 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d2a2b-66f5-4351-b65a-31c50618208b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:00.725038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 28/57] 10.222.0.65 () {60 vars in 1140 bytes} [Tue May 10 19:57:00 2022] POST /identity/v3/projects => generated 298 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 19:57:02.929314 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511f1d7-7dcc-4671-90ca-8ffb9acfd228 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:02.929846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511f1d7-7dcc-4671-90ca-8ffb9acfd228 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:02.929949 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511f1d7-7dcc-4671-90ca-8ffb9acfd228 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:02.931279 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 30/58] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:02.936530 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef8eea-9dec-412f-b107-1181a7d6b61b None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:02.936914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef8eea-9dec-412f-b107-1181a7d6b61b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:02.937099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef8eea-9dec-412f-b107-1181a7d6b61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:02.953192 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-3bef8eea-9dec-412f-b107-1181a7d6b61b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:02.954419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3bef8eea-9dec-412f-b107-1181a7d6b61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:03.016607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 29/59] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:03.040829 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511f1d7-7dcc-4671-90ca-8ffb9acfd228 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:03.136469 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511f1d7-7dcc-4671-90ca-8ffb9acfd228 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:03.167961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ac2f4-b315-404e-b0cd-6cfa9e411ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:03.168736 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ac2f4-b315-404e-b0cd-6cfa9e411ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:03.169020 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ac2f4-b315-404e-b0cd-6cfa9e411ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:03.169298 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ac2f4-b315-404e-b0cd-6cfa9e411ff0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:03.170487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ac2f4-b315-404e-b0cd-6cfa9e411ff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:03.171928 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ac2f4-b315-404e-b0cd-6cfa9e411ff0 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:03.173503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 31/60] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:57:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:03.178883 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef8eea-9dec-412f-b107-1181a7d6b61b None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:03.271615 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef8eea-9dec-412f-b107-1181a7d6b61b None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:03.303592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c41a41-926d-449a-9503-984e58298b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:03.304248 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c41a41-926d-449a-9503-984e58298b37 None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:03.304537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c41a41-926d-449a-9503-984e58298b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:03.304818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c41a41-926d-449a-9503-984e58298b37 None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:03.305530 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c41a41-926d-449a-9503-984e58298b37 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:03.307003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c41a41-926d-449a-9503-984e58298b37 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:03.403569 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 30/61] 10.222.0.65 () {60 vars in 1132 bytes} [Tue May 10 19:57:03 2022] POST /identity/v3/users => generated 275 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:57:06.672444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ebf7c-4cae-47a1-a15b-60475086bf3b None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:06.673306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ebf7c-4cae-47a1-a15b-60475086bf3b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:06.673725 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ebf7c-4cae-47a1-a15b-60475086bf3b None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:06.675255 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 32/62] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:06.680510 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc3066-4951-42c1-b2c6-2022a0279533 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:06.680921 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc3066-4951-42c1-b2c6-2022a0279533 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:06.681083 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc3066-4951-42c1-b2c6-2022a0279533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:06.702309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-77fc3066-4951-42c1-b2c6-2022a0279533 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:06.703519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-77fc3066-4951-42c1-b2c6-2022a0279533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:06.795450 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 31/63] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:06.830900 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ebf7c-4cae-47a1-a15b-60475086bf3b None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:06.961109 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ebf7c-4cae-47a1-a15b-60475086bf3b None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:07.008213 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72714e2b-a14a-460a-8624-3bc114af513b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:07.009296 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72714e2b-a14a-460a-8624-3bc114af513b None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:07.009524 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93c90d-f264-4dae-b7fc-9ea64f0dddae None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:07.009784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72714e2b-a14a-460a-8624-3bc114af513b None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:07.010063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72714e2b-a14a-460a-8624-3bc114af513b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:07.010425 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93c90d-f264-4dae-b7fc-9ea64f0dddae None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:07.010873 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93c90d-f264-4dae-b7fc-9ea64f0dddae None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:07.012846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 32/64] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:07.017556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b987b3-5faf-4323-bd3f-4473f45daa49 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:07.018071 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b987b3-5faf-4323-bd3f-4473f45daa49 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:07.018671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b987b3-5faf-4323-bd3f-4473f45daa49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:07.035683 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72714e2b-a14a-460a-8624-3bc114af513b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:07.043761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-72714e2b-a14a-460a-8624-3bc114af513b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:07.043761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:07.043761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:07.043761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:07.043761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:07.043761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:07.043761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:07.043761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:07.043761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:07.043761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:07.043761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:07.043761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:07.043761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:07.043761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:07.043761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:07.043761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:07.044613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:07.044613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:07.044613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:07.044613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:07.044613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:07.044613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:07.044613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:07.044613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:07.044613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:07.044613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:07.044613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:07.044613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:07.044613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:07.044613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:07.044613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:07.044613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:07.045366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:07.045366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:07.045366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:07.045366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:07.045366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:07.045366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:07.045366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:07.045366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:07.045366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:07.045366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:07.045366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:07.045366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:07.048101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 33/65] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:07.049278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-19b987b3-5faf-4323-bd3f-4473f45daa49 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:07.050694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-19b987b3-5faf-4323-bd3f-4473f45daa49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:07.053284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72714e2b-a14a-460a-8624-3bc114af513b None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:07.141880 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 33/66] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:07.143295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b764898-5dd0-4193-ac9a-fb2ae5d8d281 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:07.143893 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b764898-5dd0-4193-ac9a-fb2ae5d8d281 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:07.144192 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b764898-5dd0-4193-ac9a-fb2ae5d8d281 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:07.145877 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 34/67] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:07.147050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1915c52-5d58-4299-bb63-887e3e2c98f0 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:07.150152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1915c52-5d58-4299-bb63-887e3e2c98f0 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:07.150666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1915c52-5d58-4299-bb63-887e3e2c98f0 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:07.152186 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 35/68] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:07.153514 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39e533-2f83-4fe3-a468-3aeb540d643f None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:07.153798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39e533-2f83-4fe3-a468-3aeb540d643f None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:07.154074 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39e533-2f83-4fe3-a468-3aeb540d643f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:07.171508 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72714e2b-a14a-460a-8624-3bc114af513b None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:07.173847 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-ea39e533-2f83-4fe3-a468-3aeb540d643f None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:07.175065 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-ea39e533-2f83-4fe3-a468-3aeb540d643f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:07.205462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d49e89-5060-4bb1-bf92-ab818aedd116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:07.206158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d49e89-5060-4bb1-bf92-ab818aedd116 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:07.206488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d49e89-5060-4bb1-bf92-ab818aedd116 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:07.206765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d49e89-5060-4bb1-bf92-ab818aedd116 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:07.207514 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d49e89-5060-4bb1-bf92-ab818aedd116 None admin] RBAC: Adding query filter params (name=member) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:07.207902 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d49e89-5060-4bb1-bf92-ab818aedd116 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:07.209201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d49e89-5060-4bb1-bf92-ab818aedd116 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:07.218254 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 34/69] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:07.219517 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a42129-4846-430d-8b27-3349d4da9da5 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:07.219815 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a42129-4846-430d-8b27-3349d4da9da5 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:07.220099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a42129-4846-430d-8b27-3349d4da9da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:07.236904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-16a42129-4846-430d-8b27-3349d4da9da5 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:07.238119 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-16a42129-4846-430d-8b27-3349d4da9da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:07.247051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 36/70] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:07.247722 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39e533-2f83-4fe3-a468-3aeb540d643f None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:07.309317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 35/71] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:07.310100 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a42129-4846-430d-8b27-3349d4da9da5 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:07.348309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39e533-2f83-4fe3-a468-3aeb540d643f None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:07.381902 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6965730-a587-42dd-8011-aa846242e009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:07.383336 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6965730-a587-42dd-8011-aa846242e009 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:07.383628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6965730-a587-42dd-8011-aa846242e009 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:07.383907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6965730-a587-42dd-8011-aa846242e009 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:07.397109 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6965730-a587-42dd-8011-aa846242e009 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:07.406779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-b6965730-a587-42dd-8011-aa846242e009 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:07.406779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:07.406779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:07.406779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:07.406779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:07.406779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:07.406779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:07.406779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:07.406779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:07.406779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:07.406779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:07.406779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:07.406779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:07.406779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:07.406779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:07.406779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:07.407767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:07.407767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:07.407767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:07.407767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:07.407767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:07.407767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:07.407767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:07.407767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:07.407767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:07.407767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:07.407767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:07.407767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:07.407767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:07.407767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:07.407767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:07.407767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:07.408528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:07.408528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:07.408528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:07.408528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:07.408528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:07.408528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:07.408528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:07.408528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:07.408528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:07.408528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:07.408528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:07.408528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:07.408528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 37/72] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:57:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:07.413832 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6965730-a587-42dd-8011-aa846242e009 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:07.424414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a42129-4846-430d-8b27-3349d4da9da5 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:07.465075 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbfdb1-1ac9-46f4-94f4-dfe9c7b2a70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:07.466463 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbfdb1-1ac9-46f4-94f4-dfe9c7b2a70b None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:07.466760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbfdb1-1ac9-46f4-94f4-dfe9c7b2a70b None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:07.467046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbfdb1-1ac9-46f4-94f4-dfe9c7b2a70b None admin] PATH_INFO: `/v3/users/13e9677bc30f49ee88a08bd637b062b2` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:07.481549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbfdb1-1ac9-46f4-94f4-dfe9c7b2a70b None admin] RBAC: Authorizing `identity:get_user(user_id=13e9677bc30f49ee88a08bd637b062b2)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:07.483312 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbfdb1-1ac9-46f4-94f4-dfe9c7b2a70b None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:07.485218 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 36/73] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:07 2022] GET /identity/v3/users/13e9677bc30f49ee88a08bd637b062b2 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:07.486234 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbfdb1-1ac9-46f4-94f4-dfe9c7b2a70b None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:07.517732 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6965730-a587-42dd-8011-aa846242e009 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:07.553849 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b9663-6c98-4143-8136-7f21f36cc5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:07.555188 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b9663-6c98-4143-8136-7f21f36cc5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:07.555481 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b9663-6c98-4143-8136-7f21f36cc5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:07.555777 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b9663-6c98-4143-8136-7f21f36cc5b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:07.571437 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b9663-6c98-4143-8136-7f21f36cc5b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:07.579183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-882b9663-6c98-4143-8136-7f21f36cc5b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:07.579183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:07.579183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:07.579183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:07.579183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:07.579183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:07.579183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:07.579183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:07.579183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:07.579183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:07.579183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:07.579183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:07.579183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:07.579183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:07.579183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:07.579183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:07.580051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:07.580051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:07.580051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:07.580051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:07.580051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:07.580051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:07.580051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:07.580051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:07.580051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:07.580051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:07.580051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:07.580051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:07.580051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:07.580051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:07.580051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:07.580051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:07.582016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:07.582016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:07.582016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:07.582016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:07.582016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:07.582016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:07.582016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:07.582016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:07.582016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:07.582016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:07.582016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:07.582016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:07.582016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 38/74] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:07.582016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b9663-6c98-4143-8136-7f21f36cc5b9 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:07.598866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbfdb1-1ac9-46f4-94f4-dfe9c7b2a70b None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:07.631757 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8af272-eac6-42c4-aff1-ba7af7f7e6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:07.632572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8af272-eac6-42c4-aff1-ba7af7f7e6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:07.632857 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8af272-eac6-42c4-aff1-ba7af7f7e6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:07.633140 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8af272-eac6-42c4-aff1-ba7af7f7e6d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:07.642784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8af272-eac6-42c4-aff1-ba7af7f7e6d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:07.650426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-5f8af272-eac6-42c4-aff1-ba7af7f7e6d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:07.650426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:07.650426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:07.650426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:07.650426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:07.650426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:07.650426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:07.650426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:07.650426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:07.650426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:07.650426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:07.650426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:07.650426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:07.650426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:07.650426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:07.650426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:07.651248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:07.651248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:07.651248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:07.651248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:07.651248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:07.651248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:07.651248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:07.651248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:07.651248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:07.651248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:07.651248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:07.651248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:07.651248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:07.651248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:07.651248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:07.651248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:07.652015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:07.652015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:07.652015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:07.652015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:07.652015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:07.652015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:07.652015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:07.652015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:07.652015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:07.652015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:07.652015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:07.652015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:07.652015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 37/75] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:57:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:07.652712 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8af272-eac6-42c4-aff1-ba7af7f7e6d1 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:07.676800 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b9663-6c98-4143-8136-7f21f36cc5b9 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:07.708292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5da957-b69d-43f2-96c4-2ed70b432d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:07.708955 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5da957-b69d-43f2-96c4-2ed70b432d83 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:07.709245 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5da957-b69d-43f2-96c4-2ed70b432d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:07.709533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5da957-b69d-43f2-96c4-2ed70b432d83 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:07.710353 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5da957-b69d-43f2-96c4-2ed70b432d83 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:07.710665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5da957-b69d-43f2-96c4-2ed70b432d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:07.712041 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5da957-b69d-43f2-96c4-2ed70b432d83 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:07.719583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 39/76] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:57:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:07.721612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805e0f8-060f-4d95-bd99-fc7547b8718c None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:07.721901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805e0f8-060f-4d95-bd99-fc7547b8718c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:07.722209 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805e0f8-060f-4d95-bd99-fc7547b8718c None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:07.723622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 40/77] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:07.724261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805e0f8-060f-4d95-bd99-fc7547b8718c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:07.751314 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8af272-eac6-42c4-aff1-ba7af7f7e6d1 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:07.782070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4d3bc-4769-46b6-82ef-4accb3b30906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:07.782752 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4d3bc-4769-46b6-82ef-4accb3b30906 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:07.783017 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4d3bc-4769-46b6-82ef-4accb3b30906 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:07.783290 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4d3bc-4769-46b6-82ef-4accb3b30906 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:07.819591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805e0f8-060f-4d95-bd99-fc7547b8718c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:07.846024 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4d3bc-4769-46b6-82ef-4accb3b30906 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:07.847513 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4d3bc-4769-46b6-82ef-4accb3b30906 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:07.852746 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308707e0-816a-40f5-808b-c3d340124873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:07.853402 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308707e0-816a-40f5-808b-c3d340124873 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:07.853684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308707e0-816a-40f5-808b-c3d340124873 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:07.853959 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308707e0-816a-40f5-808b-c3d340124873 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:07.854731 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308707e0-816a-40f5-808b-c3d340124873 None admin] RBAC: Adding query filter params (name=member) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:07.855037 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308707e0-816a-40f5-808b-c3d340124873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:07.856389 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308707e0-816a-40f5-808b-c3d340124873 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:07.864519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 41/78] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:07.865286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308707e0-816a-40f5-808b-c3d340124873 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:07.948906 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 38/79] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:07.950046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4d3bc-4769-46b6-82ef-4accb3b30906 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:07.959574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308707e0-816a-40f5-808b-c3d340124873 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:07.992093 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f55bc2-ef62-4e30-bf26-117e9b1e6875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:07.992760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f55bc2-ef62-4e30-bf26-117e9b1e6875 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:07.993046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f55bc2-ef62-4e30-bf26-117e9b1e6875 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:07.993323 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f55bc2-ef62-4e30-bf26-117e9b1e6875 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:07.994358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f55bc2-ef62-4e30-bf26-117e9b1e6875 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:07.994670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f55bc2-ef62-4e30-bf26-117e9b1e6875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:07.996043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f55bc2-ef62-4e30-bf26-117e9b1e6875 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:08.003760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 42/80] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:08.004865 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6758a-8191-4846-a414-f1bb5c905cd5 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:08.005147 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6758a-8191-4846-a414-f1bb5c905cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:08.005424 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6758a-8191-4846-a414-f1bb5c905cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:08.020626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-f4b6758a-8191-4846-a414-f1bb5c905cd5 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:08.021805 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-f4b6758a-8191-4846-a414-f1bb5c905cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:08.043174 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4d3bc-4769-46b6-82ef-4accb3b30906 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:08.074627 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1946c-702f-40d8-932b-7c39fd1150cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:08.075488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1946c-702f-40d8-932b-7c39fd1150cb None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:08.075799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1946c-702f-40d8-932b-7c39fd1150cb None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:08.076079 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1946c-702f-40d8-932b-7c39fd1150cb None admin] PATH_INFO: `/v3/users/13e9677bc30f49ee88a08bd637b062b2` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:08.077309 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1946c-702f-40d8-932b-7c39fd1150cb None admin] RBAC: Authorizing `identity:get_user(user_id=13e9677bc30f49ee88a08bd637b062b2)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:08.078785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1946c-702f-40d8-932b-7c39fd1150cb None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:08.080406 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 39/81] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:07 2022] GET /identity/v3/users/13e9677bc30f49ee88a08bd637b062b2 => generated 275 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:08.081159 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1946c-702f-40d8-932b-7c39fd1150cb None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:08.085076 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 43/82] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:08.086664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6758a-8191-4846-a414-f1bb5c905cd5 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:08.178583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6758a-8191-4846-a414-f1bb5c905cd5 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:08.178772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1946c-702f-40d8-932b-7c39fd1150cb None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:08.214136 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e13b3c-73e8-4104-9db9-42c8c607399e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:08.215018 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e13b3c-73e8-4104-9db9-42c8c607399e None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:08.215302 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e13b3c-73e8-4104-9db9-42c8c607399e None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:08.215579 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e13b3c-73e8-4104-9db9-42c8c607399e None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:08.215668 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607b826-f051-42e8-9aa6-0cf937eb2f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:08.216451 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607b826-f051-42e8-9aa6-0cf937eb2f39 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:08.216733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607b826-f051-42e8-9aa6-0cf937eb2f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:08.217008 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607b826-f051-42e8-9aa6-0cf937eb2f39 None admin] PATH_INFO: `/v3/users/13e9677bc30f49ee88a08bd637b062b2` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:08.217092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e13b3c-73e8-4104-9db9-42c8c607399e None admin] RBAC: Authorizing `identity:get_project(project_id=675b1b490743424e9d44dd2e484dfa95)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:08.218238 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607b826-f051-42e8-9aa6-0cf937eb2f39 None admin] RBAC: Authorizing `identity:get_user(user_id=13e9677bc30f49ee88a08bd637b062b2)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:08.218518 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e13b3c-73e8-4104-9db9-42c8c607399e None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:08.219672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607b826-f051-42e8-9aa6-0cf937eb2f39 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:08.220216 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 44/83] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:08 2022] GET /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95 => generated 298 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:08.220983 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e13b3c-73e8-4104-9db9-42c8c607399e None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:08.221306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 40/84] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:08 2022] GET /identity/v3/users/13e9677bc30f49ee88a08bd637b062b2 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:08.222062 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607b826-f051-42e8-9aa6-0cf937eb2f39 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:08.316622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607b826-f051-42e8-9aa6-0cf937eb2f39 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:08.323254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e13b3c-73e8-4104-9db9-42c8c607399e None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:08.348732 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3892fa-187c-40af-bb55-09731fff6818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:08.349379 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3892fa-187c-40af-bb55-09731fff6818 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:08.349666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3892fa-187c-40af-bb55-09731fff6818 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:08.349944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3892fa-187c-40af-bb55-09731fff6818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:08.355889 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750dcfe9-295b-41e3-924e-15902fee0a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:08.356751 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dcfe9-295b-41e3-924e-15902fee0a6a None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:08.357044 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dcfe9-295b-41e3-924e-15902fee0a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:08.357326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dcfe9-295b-41e3-924e-15902fee0a6a None admin] PATH_INFO: `/v3/users/729fa49ea5f943b2b09c35b636e34ee5` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:08.358805 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750dcfe9-295b-41e3-924e-15902fee0a6a None admin] RBAC: Authorizing `identity:get_user(user_id=729fa49ea5f943b2b09c35b636e34ee5)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:08.360257 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750dcfe9-295b-41e3-924e-15902fee0a6a None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:08.361904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 45/85] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:08 2022] GET /identity/v3/users/729fa49ea5f943b2b09c35b636e34ee5 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:08.362694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750dcfe9-295b-41e3-924e-15902fee0a6a None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:08.413566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3892fa-187c-40af-bb55-09731fff6818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:08.415039 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3892fa-187c-40af-bb55-09731fff6818 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:08.459739 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750dcfe9-295b-41e3-924e-15902fee0a6a None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:08.491139 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8a590-0d8a-4df9-bd22-8c6b3ea5175e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:08.491985 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8a590-0d8a-4df9-bd22-8c6b3ea5175e None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:08.492281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8a590-0d8a-4df9-bd22-8c6b3ea5175e None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:08.492560 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8a590-0d8a-4df9-bd22-8c6b3ea5175e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:08.493989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8a590-0d8a-4df9-bd22-8c6b3ea5175e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:08.495489 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8a590-0d8a-4df9-bd22-8c6b3ea5175e None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:08.497099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 46/86] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:57:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:08.497863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8a590-0d8a-4df9-bd22-8c6b3ea5175e None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:08.507739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 41/87] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:08.508583 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3892fa-187c-40af-bb55-09731fff6818 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:08.592397 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8a590-0d8a-4df9-bd22-8c6b3ea5175e None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:08.602524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3892fa-187c-40af-bb55-09731fff6818 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:08.623307 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae858ca5-1ce5-42f0-a4ab-8bf656b03485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:08.623959 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae858ca5-1ce5-42f0-a4ab-8bf656b03485 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:08.624244 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae858ca5-1ce5-42f0-a4ab-8bf656b03485 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:08.624520 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae858ca5-1ce5-42f0-a4ab-8bf656b03485 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:08.625436 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae858ca5-1ce5-42f0-a4ab-8bf656b03485 None admin] RBAC: Adding query filter params (role.id=5f62ce25636443f8a5add67f208afe62, scope.project.id=675b1b490743424e9d44dd2e484dfa95, user.id=13e9677bc30f49ee88a08bd637b062b2) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:08.625750 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae858ca5-1ce5-42f0-a4ab-8bf656b03485 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:08.627205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae858ca5-1ce5-42f0-a4ab-8bf656b03485 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:08.634454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d4dcc-8394-412c-b253-62a321628b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:08.635097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d4dcc-8394-412c-b253-62a321628b15 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:08.635383 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d4dcc-8394-412c-b253-62a321628b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:08.635660 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d4dcc-8394-412c-b253-62a321628b15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:08.635796 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 47/88] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:57:08 2022] GET /identity/v3/role_assignments?user.id=13e9677bc30f49ee88a08bd637b062b2&scope.project.id=675b1b490743424e9d44dd2e484dfa95&role.id=5f62ce25636443f8a5add67f208afe62 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:08.636582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae858ca5-1ce5-42f0-a4ab-8bf656b03485 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:08.708250 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d4dcc-8394-412c-b253-62a321628b15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:08.709732 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d4dcc-8394-412c-b253-62a321628b15 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:08.741254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae858ca5-1ce5-42f0-a4ab-8bf656b03485 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:08.775891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e0883-2868-4769-98c3-618bbb818ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:08.776565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e0883-2868-4769-98c3-618bbb818ebc None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:08.776898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e0883-2868-4769-98c3-618bbb818ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:08.777125 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e0883-2868-4769-98c3-618bbb818ebc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:08.816825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 42/89] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:08.818028 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d4dcc-8394-412c-b253-62a321628b15 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:08.845826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e0883-2868-4769-98c3-618bbb818ebc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:08.847146 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e0883-2868-4769-98c3-618bbb818ebc None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:08.916396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d4dcc-8394-412c-b253-62a321628b15 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:08.947536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 48/90] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:08.947536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e0883-2868-4769-98c3-618bbb818ebc None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:08.949078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac12c20-70e4-40a1-8785-498d967fe275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:08.949747 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac12c20-70e4-40a1-8785-498d967fe275 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:08.950029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac12c20-70e4-40a1-8785-498d967fe275 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:08.950333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac12c20-70e4-40a1-8785-498d967fe275 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:08.951049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac12c20-70e4-40a1-8785-498d967fe275 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:08.952496 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac12c20-70e4-40a1-8785-498d967fe275 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:09.039028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e0883-2868-4769-98c3-618bbb818ebc None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:09.049038 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 43/91] 10.222.0.65 () {60 vars in 1132 bytes} [Tue May 10 19:57:08 2022] POST /identity/v3/users => generated 289 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:57:09.049694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac12c20-70e4-40a1-8785-498d967fe275 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:09.077067 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77147778-97d9-4023-988f-d7c3a9a8d4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:09.078257 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77147778-97d9-4023-988f-d7c3a9a8d4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:09.078446 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77147778-97d9-4023-988f-d7c3a9a8d4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:09.078732 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77147778-97d9-4023-988f-d7c3a9a8d4f2 None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:09.080095 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77147778-97d9-4023-988f-d7c3a9a8d4f2 None admin] RBAC: Authorizing `identity:get_project(project_id=675b1b490743424e9d44dd2e484dfa95)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:09.081709 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77147778-97d9-4023-988f-d7c3a9a8d4f2 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:09.084167 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 49/92] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:08 2022] GET /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:09.084733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77147778-97d9-4023-988f-d7c3a9a8d4f2 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:09.174059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac12c20-70e4-40a1-8785-498d967fe275 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:09.195062 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77147778-97d9-4023-988f-d7c3a9a8d4f2 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:09.213407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35083d3-e7fe-400b-9bbb-41cc78021c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:09.214799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35083d3-e7fe-400b-9bbb-41cc78021c6f None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:09.215335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35083d3-e7fe-400b-9bbb-41cc78021c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:09.215802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35083d3-e7fe-400b-9bbb-41cc78021c6f None admin] PATH_INFO: `/v3/projects/9c647a719b03434cbdff743293674518` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:09.217346 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35083d3-e7fe-400b-9bbb-41cc78021c6f None admin] RBAC: Authorizing `identity:get_project(project_id=9c647a719b03434cbdff743293674518)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:09.219198 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35083d3-e7fe-400b-9bbb-41cc78021c6f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:09.221588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 44/93] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:09 2022] GET /identity/v3/projects/9c647a719b03434cbdff743293674518 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:09.222755 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35083d3-e7fe-400b-9bbb-41cc78021c6f None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:09.232756 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74ea20-4601-4601-9c4c-e0d3f765f8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:09.233759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74ea20-4601-4601-9c4c-e0d3f765f8a1 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:09.234053 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74ea20-4601-4601-9c4c-e0d3f765f8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:09.234364 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74ea20-4601-4601-9c4c-e0d3f765f8a1 None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:09.235642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74ea20-4601-4601-9c4c-e0d3f765f8a1 None admin] RBAC: Authorizing `identity:get_project(project_id=675b1b490743424e9d44dd2e484dfa95)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:09.237252 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74ea20-4601-4601-9c4c-e0d3f765f8a1 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:09.243639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 50/94] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:09 2022] GET /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:09.243639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74ea20-4601-4601-9c4c-e0d3f765f8a1 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:09.343427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35083d3-e7fe-400b-9bbb-41cc78021c6f None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:09.344080 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74ea20-4601-4601-9c4c-e0d3f765f8a1 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:09.378352 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b77fb-78ce-495d-b0f4-c42adfcabc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:09.379127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b77fb-78ce-495d-b0f4-c42adfcabc7b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:09.379448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe295d-f4ab-451c-a2a6-42d6f5adb3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:09.379574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b77fb-78ce-495d-b0f4-c42adfcabc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:09.379931 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b77fb-78ce-495d-b0f4-c42adfcabc7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:09.380493 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe295d-f4ab-451c-a2a6-42d6f5adb3cb None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:09.380918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe295d-f4ab-451c-a2a6-42d6f5adb3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:09.381035 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b77fb-78ce-495d-b0f4-c42adfcabc7b None admin] RBAC: Adding query filter params (role.id=5f62ce25636443f8a5add67f208afe62, scope.project.id=9c647a719b03434cbdff743293674518, user.id=13e9677bc30f49ee88a08bd637b062b2) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:09.381430 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe295d-f4ab-451c-a2a6-42d6f5adb3cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:09.381547 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b77fb-78ce-495d-b0f4-c42adfcabc7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:09.383071 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe295d-f4ab-451c-a2a6-42d6f5adb3cb None admin] RBAC: Adding query filter params (role.id=4fbb0a346f5f4f36bb32664ee799db42, scope.project.id=675b1b490743424e9d44dd2e484dfa95, user.id=13e9677bc30f49ee88a08bd637b062b2) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:09.383550 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe295d-f4ab-451c-a2a6-42d6f5adb3cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:09.383787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b77fb-78ce-495d-b0f4-c42adfcabc7b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:09.385274 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe295d-f4ab-451c-a2a6-42d6f5adb3cb None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:09.390809 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 51/95] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:57:09 2022] GET /identity/v3/role_assignments?user.id=13e9677bc30f49ee88a08bd637b062b2&scope.project.id=9c647a719b03434cbdff743293674518&role.id=5f62ce25636443f8a5add67f208afe62 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:09.390972 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b77fb-78ce-495d-b0f4-c42adfcabc7b None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:09.392325 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 45/96] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:57:09 2022] GET /identity/v3/role_assignments?user.id=13e9677bc30f49ee88a08bd637b062b2&scope.project.id=675b1b490743424e9d44dd2e484dfa95&role.id=4fbb0a346f5f4f36bb32664ee799db42 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:09.511390 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b77fb-78ce-495d-b0f4-c42adfcabc7b None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:09.549904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f332b7-dcf5-4112-89b8-3ca292bfc45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:09.550616 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f332b7-dcf5-4112-89b8-3ca292bfc45e None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:09.550902 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f332b7-dcf5-4112-89b8-3ca292bfc45e None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:09.551179 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f332b7-dcf5-4112-89b8-3ca292bfc45e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:09.552052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f332b7-dcf5-4112-89b8-3ca292bfc45e None admin] RBAC: Adding query filter params (role.id=5c87e804d0764309ba08ad6e06b23a87, scope.project.id=675b1b490743424e9d44dd2e484dfa95, user.id=729fa49ea5f943b2b09c35b636e34ee5) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:09.552368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f332b7-dcf5-4112-89b8-3ca292bfc45e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:09.554404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f332b7-dcf5-4112-89b8-3ca292bfc45e None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:09.561336 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 52/97] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:57:09 2022] GET /identity/v3/role_assignments?user.id=729fa49ea5f943b2b09c35b636e34ee5&scope.project.id=675b1b490743424e9d44dd2e484dfa95&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:11.178695 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e26b95-445e-4541-8396-56bd4790e4b5 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:11.179729 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e26b95-445e-4541-8396-56bd4790e4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:11.179802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e26b95-445e-4541-8396-56bd4790e4b5 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:11.181138 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 46/98] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:11.186347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11372663-fd8c-4a3d-8dbe-ba5fde83938e None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:11.186749 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11372663-fd8c-4a3d-8dbe-ba5fde83938e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:11.186920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11372663-fd8c-4a3d-8dbe-ba5fde83938e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:11.211131 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-11372663-fd8c-4a3d-8dbe-ba5fde83938e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:11.212312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-11372663-fd8c-4a3d-8dbe-ba5fde83938e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:11.300367 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 53/99] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:11.334196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e26b95-445e-4541-8396-56bd4790e4b5 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:11.459762 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e26b95-445e-4541-8396-56bd4790e4b5 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:11.503461 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5af71c-843f-4d9e-a38f-614817df55dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:11.504423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5af71c-843f-4d9e-a38f-614817df55dd None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:11.504705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5af71c-843f-4d9e-a38f-614817df55dd None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:11.504976 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5af71c-843f-4d9e-a38f-614817df55dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:11.522969 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5af71c-843f-4d9e-a38f-614817df55dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:11.530692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-af5af71c-843f-4d9e-a38f-614817df55dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:11.530692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:11.530692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:11.530692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:11.530692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:11.530692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:11.530692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:11.530692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:11.530692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:11.530692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:11.530692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:11.530692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:11.530692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:11.530692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:11.530692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:11.530692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:11.531548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:11.531548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:11.531548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:11.531548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:11.531548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:11.531548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:11.531548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:11.531548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:11.531548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:11.531548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:11.531548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:11.531548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:11.531548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:11.531548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:11.531548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:11.531548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:11.532366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:11.532366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:11.532366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:11.532366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:11.532366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:11.532366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:11.532366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:11.532366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:11.532366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:11.532366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:11.532366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:11.532366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:11.532366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 47/100] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:57:11.537865 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11372663-fd8c-4a3d-8dbe-ba5fde83938e None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:11.641854 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11372663-fd8c-4a3d-8dbe-ba5fde83938e None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:11.676290 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f368769-1dfd-49c0-93cf-d47e5b5620ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:11.677327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f368769-1dfd-49c0-93cf-d47e5b5620ce None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:11.677810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f368769-1dfd-49c0-93cf-d47e5b5620ce None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:11.678302 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f368769-1dfd-49c0-93cf-d47e5b5620ce None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:11.679812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f368769-1dfd-49c0-93cf-d47e5b5620ce None admin] RBAC: Adding query filter params (name=member) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:11.680436 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f368769-1dfd-49c0-93cf-d47e5b5620ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:11.682148 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f368769-1dfd-49c0-93cf-d47e5b5620ce None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:11.690660 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 54/101] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:11.696364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5af71c-843f-4d9e-a38f-614817df55dd None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:11.852288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5af71c-843f-4d9e-a38f-614817df55dd None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:11.898760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf1e98-2c80-42b3-8f0c-6c3435fc95f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:11.899631 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf1e98-2c80-42b3-8f0c-6c3435fc95f6 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:11.899914 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf1e98-2c80-42b3-8f0c-6c3435fc95f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:11.900190 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf1e98-2c80-42b3-8f0c-6c3435fc95f6 None admin] PATH_INFO: `/v3/users/13e9677bc30f49ee88a08bd637b062b2` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:11.906899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf1e98-2c80-42b3-8f0c-6c3435fc95f6 None admin] RBAC: Authorizing `identity:get_user(user_id=13e9677bc30f49ee88a08bd637b062b2)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:11.908391 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf1e98-2c80-42b3-8f0c-6c3435fc95f6 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:11.910044 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 48/102] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:11 2022] GET /identity/v3/users/13e9677bc30f49ee88a08bd637b062b2 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:11.916539 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f368769-1dfd-49c0-93cf-d47e5b5620ce None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:12.046221 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f368769-1dfd-49c0-93cf-d47e5b5620ce None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:12.079849 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e54b08-e568-41e0-b037-028120e650a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:12.081075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e54b08-e568-41e0-b037-028120e650a7 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:12.081559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e54b08-e568-41e0-b037-028120e650a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:12.082033 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e54b08-e568-41e0-b037-028120e650a7 None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:12.083461 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e54b08-e568-41e0-b037-028120e650a7 None admin] RBAC: Authorizing `identity:get_project(project_id=675b1b490743424e9d44dd2e484dfa95)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:12.085691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e54b08-e568-41e0-b037-028120e650a7 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:12.087769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 55/103] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:11 2022] GET /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:12.093570 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf1e98-2c80-42b3-8f0c-6c3435fc95f6 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:12.271909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf1e98-2c80-42b3-8f0c-6c3435fc95f6 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:12.329866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9fcce-9c62-471d-bd41-47df315cfc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:12.330730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9fcce-9c62-471d-bd41-47df315cfc80 None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:12.331013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9fcce-9c62-471d-bd41-47df315cfc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:12.331287 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9fcce-9c62-471d-bd41-47df315cfc80 None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95/users/13e9677bc30f49ee88a08bd637b062b2/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:12.333475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea9fcce-9c62-471d-bd41-47df315cfc80 None admin] RBAC: Authorizing `identity:create_grant(project_id=675b1b490743424e9d44dd2e484dfa95, user_id=13e9677bc30f49ee88a08bd637b062b2, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:12.335578 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea9fcce-9c62-471d-bd41-47df315cfc80 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:12.424807 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 49/104] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:57:12 2022] PUT /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95/users/13e9677bc30f49ee88a08bd637b062b2/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:57:12.644058 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf87b00-072f-4a8e-b6b3-836b7aff6cad None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:12.645079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf87b00-072f-4a8e-b6b3-836b7aff6cad None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:12.645586 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf87b00-072f-4a8e-b6b3-836b7aff6cad None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:12.647282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 56/105] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:12.665587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a18ce-3be0-451a-8702-0a7e23008959 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:12.666278 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a18ce-3be0-451a-8702-0a7e23008959 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:12.666788 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a18ce-3be0-451a-8702-0a7e23008959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:12.691433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-086a18ce-3be0-451a-8702-0a7e23008959 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:12.693008 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-086a18ce-3be0-451a-8702-0a7e23008959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:12.748169 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30510d3-3e39-469d-a953-8b79963fd4ba None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:12.748815 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30510d3-3e39-469d-a953-8b79963fd4ba None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:12.749313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30510d3-3e39-469d-a953-8b79963fd4ba None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:12.752238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 57/106] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:12.759992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d032b4-4836-4630-b539-4a9fc36aa4c8 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:12.760615 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d032b4-4836-4630-b539-4a9fc36aa4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:12.761098 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d032b4-4836-4630-b539-4a9fc36aa4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:12.781465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-b8d032b4-4836-4630-b539-4a9fc36aa4c8 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:12.783678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b8d032b4-4836-4630-b539-4a9fc36aa4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:12.808009 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 50/107] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:12.833769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a18ce-3be0-451a-8702-0a7e23008959 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:12.866117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 58/108] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:57:12.890390 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d032b4-4836-4630-b539-4a9fc36aa4c8 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:12.949176 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a18ce-3be0-451a-8702-0a7e23008959 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:12.984975 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eaafba-222c-47b4-99b8-117c09bbbec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:12.985775 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaafba-222c-47b4-99b8-117c09bbbec1 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:12.986058 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaafba-222c-47b4-99b8-117c09bbbec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:12.986363 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaafba-222c-47b4-99b8-117c09bbbec1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:12.997855 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eaafba-222c-47b4-99b8-117c09bbbec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:13.005853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-04eaafba-222c-47b4-99b8-117c09bbbec1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:13.005853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:13.005853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:13.005853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:13.005853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:13.005853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:13.005853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:13.005853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:13.005853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:13.005853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:13.005853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:13.005853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:13.005853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:13.005853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:13.005853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:13.005853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:13.007085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:13.007085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:13.007085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:13.007085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:13.007085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:13.007085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:13.007085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:13.007085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:13.007085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:13.007085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:13.007085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:13.007085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:13.007085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:13.007085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:13.007085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:13.007085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:13.007868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:13.007868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:13.007868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:13.007868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:13.007868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:13.007868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:13.007868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:13.007868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:13.007868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:13.007868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:13.007868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:13.007868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:13.008472 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d032b4-4836-4630-b539-4a9fc36aa4c8 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:13.010378 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 51/109] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:57:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:13.012433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f753247-a577-4218-bbaa-3bb939537a30 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:13.012973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f753247-a577-4218-bbaa-3bb939537a30 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:13.013462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f753247-a577-4218-bbaa-3bb939537a30 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:13.015163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 52/110] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:13.016822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923c5c8-07fc-4498-9e04-0406c7e17b36 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:13.017329 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923c5c8-07fc-4498-9e04-0406c7e17b36 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:13.017796 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923c5c8-07fc-4498-9e04-0406c7e17b36 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:13.019569 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 53/111] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:13.028117 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5b93c-3ae1-4280-835c-e9fdcb3b4a39 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:13.028117 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5b93c-3ae1-4280-835c-e9fdcb3b4a39 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:13.028117 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5b93c-3ae1-4280-835c-e9fdcb3b4a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:13.049448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-83a5b93c-3ae1-4280-835c-e9fdcb3b4a39 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:13.050808 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-83a5b93c-3ae1-4280-835c-e9fdcb3b4a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:13.051473 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49507706-8d28-4a04-bdf8-03044ee87ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:13.052355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49507706-8d28-4a04-bdf8-03044ee87ffe None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:13.052711 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49507706-8d28-4a04-bdf8-03044ee87ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:13.053053 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49507706-8d28-4a04-bdf8-03044ee87ffe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:13.064519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49507706-8d28-4a04-bdf8-03044ee87ffe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:13.072574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-49507706-8d28-4a04-bdf8-03044ee87ffe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:13.072574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:13.072574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:13.072574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:13.072574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:13.072574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:13.072574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:13.072574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:13.072574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:13.072574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:13.072574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:13.072574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:13.072574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:13.072574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:13.072574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:13.072574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:13.073429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:13.073429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:13.073429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:13.073429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:13.073429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:13.073429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:13.073429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:13.073429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:13.073429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:13.073429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:13.073429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:13.073429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:13.073429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:13.073429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:13.073429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:13.073429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:13.074247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:13.074247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:13.074247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:13.074247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:13.074247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:13.074247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:13.074247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:13.074247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:13.074247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:13.074247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:13.074247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:13.074247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:13.074247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 59/112] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:57:13.080370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c008572-42f5-4ae3-b962-ef93ec0a495b None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:13.080757 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c008572-42f5-4ae3-b962-ef93ec0a495b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:13.081092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c008572-42f5-4ae3-b962-ef93ec0a495b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:13.097847 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-9c008572-42f5-4ae3-b962-ef93ec0a495b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:13.099159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-9c008572-42f5-4ae3-b962-ef93ec0a495b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:13.135454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 54/113] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:13.136425 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5b93c-3ae1-4280-835c-e9fdcb3b4a39 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:13.185415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 60/114] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:13.186321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c008572-42f5-4ae3-b962-ef93ec0a495b None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:13.251897 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5b93c-3ae1-4280-835c-e9fdcb3b4a39 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:13.286584 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc533552-e679-4100-be93-529498840831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:13.288670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc533552-e679-4100-be93-529498840831 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:13.288670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc533552-e679-4100-be93-529498840831 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:13.288670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc533552-e679-4100-be93-529498840831 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:13.289173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc533552-e679-4100-be93-529498840831 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:13.289479 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc533552-e679-4100-be93-529498840831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:13.290875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc533552-e679-4100-be93-529498840831 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:13.297430 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c008572-42f5-4ae3-b962-ef93ec0a495b None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:13.303534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 55/115] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:13.303534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc533552-e679-4100-be93-529498840831 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:13.334577 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d67ce3-c47c-4151-aa30-93a9ca8ea752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:13.335753 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d67ce3-c47c-4151-aa30-93a9ca8ea752 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:13.336053 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d67ce3-c47c-4151-aa30-93a9ca8ea752 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:13.336317 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d67ce3-c47c-4151-aa30-93a9ca8ea752 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:13.337076 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d67ce3-c47c-4151-aa30-93a9ca8ea752 None admin] RBAC: Adding query filter params (name=member) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:13.337076 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d67ce3-c47c-4151-aa30-93a9ca8ea752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:13.338777 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d67ce3-c47c-4151-aa30-93a9ca8ea752 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:13.347617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 61/116] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:13.349085 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ece7a9-e12a-48f9-947e-905e9c3d2fa1 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:13.349503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ece7a9-e12a-48f9-947e-905e9c3d2fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:13.349918 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ece7a9-e12a-48f9-947e-905e9c3d2fa1 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:13.352147 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 62/117] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:13.353108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ece7a9-e12a-48f9-947e-905e9c3d2fa1 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:13.409984 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc533552-e679-4100-be93-529498840831 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:13.449839 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617dece-1af2-47e1-bc2b-a6da37432d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:13.450596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617dece-1af2-47e1-bc2b-a6da37432d1c None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:13.450887 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617dece-1af2-47e1-bc2b-a6da37432d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:13.451168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617dece-1af2-47e1-bc2b-a6da37432d1c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:13.458471 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ece7a9-e12a-48f9-947e-905e9c3d2fa1 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:13.462069 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617dece-1af2-47e1-bc2b-a6da37432d1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:13.469780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-a617dece-1af2-47e1-bc2b-a6da37432d1c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 19:57:13.469780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:13.469780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:13.469780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:13.469780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:13.469780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:13.469780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:13.469780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:13.469780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:13.469780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:13.469780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:13.469780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:13.469780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:13.469780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:13.469780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:13.469780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:13.477229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:13.477229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:13.477229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:13.477229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:13.477229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:13.477229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:13.477229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:13.477229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:13.477229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:13.477229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:13.477229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:13.477229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:13.477229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:13.477229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:13.477229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:13.477229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:13.478052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:13.478052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:13.478052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:13.478052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:13.478052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:13.478052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:13.478052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:13.478052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:13.478052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:13.478052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:13.478052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 19:57:13.478052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:13.478052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 56/118] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:13.478052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617dece-1af2-47e1-bc2b-a6da37432d1c None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:13.503977 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719be357-61a4-4c7d-a86f-d4666430790f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:13.505021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719be357-61a4-4c7d-a86f-d4666430790f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:13.505319 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719be357-61a4-4c7d-a86f-d4666430790f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:13.505649 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719be357-61a4-4c7d-a86f-d4666430790f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:13.507972 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719be357-61a4-4c7d-a86f-d4666430790f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:13.509466 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719be357-61a4-4c7d-a86f-d4666430790f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:13.511137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 63/119] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:57:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:13.516140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0801cde-41b9-4ce4-a8f7-60e9e8cb46a4 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:13.516140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0801cde-41b9-4ce4-a8f7-60e9e8cb46a4 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:13.516140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0801cde-41b9-4ce4-a8f7-60e9e8cb46a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:13.530749 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-c0801cde-41b9-4ce4-a8f7-60e9e8cb46a4 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:13.531938 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-c0801cde-41b9-4ce4-a8f7-60e9e8cb46a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:13.573789 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617dece-1af2-47e1-bc2b-a6da37432d1c None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:13.598313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 64/120] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:13.603041 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0801cde-41b9-4ce4-a8f7-60e9e8cb46a4 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:13.616773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80613e0-f8ed-40b1-9854-1317270a0ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:13.617658 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80613e0-f8ed-40b1-9854-1317270a0ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:13.617947 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80613e0-f8ed-40b1-9854-1317270a0ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:13.618274 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80613e0-f8ed-40b1-9854-1317270a0ef5 None admin] PATH_INFO: `/v3/users/729fa49ea5f943b2b09c35b636e34ee5` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:13.620020 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80613e0-f8ed-40b1-9854-1317270a0ef5 None admin] RBAC: Authorizing `identity:get_user(user_id=729fa49ea5f943b2b09c35b636e34ee5)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:13.621573 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80613e0-f8ed-40b1-9854-1317270a0ef5 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:13.623232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 57/121] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:13 2022] GET /identity/v3/users/729fa49ea5f943b2b09c35b636e34ee5 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:13.628256 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80613e0-f8ed-40b1-9854-1317270a0ef5 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:13.707526 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0801cde-41b9-4ce4-a8f7-60e9e8cb46a4 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:13.728367 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80613e0-f8ed-40b1-9854-1317270a0ef5 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:13.741155 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317dc05-c93e-4ce0-9e33-31b9a01f2e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:13.742026 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317dc05-c93e-4ce0-9e33-31b9a01f2e79 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:13.742342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317dc05-c93e-4ce0-9e33-31b9a01f2e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:13.742621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317dc05-c93e-4ce0-9e33-31b9a01f2e79 None admin] PATH_INFO: `/v3/users/13e9677bc30f49ee88a08bd637b062b2` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:13.744079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6317dc05-c93e-4ce0-9e33-31b9a01f2e79 None admin] RBAC: Authorizing `identity:get_user(user_id=13e9677bc30f49ee88a08bd637b062b2)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:13.745529 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6317dc05-c93e-4ce0-9e33-31b9a01f2e79 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:13.751558 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 65/122] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:13 2022] GET /identity/v3/users/13e9677bc30f49ee88a08bd637b062b2 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:13.751558 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317dc05-c93e-4ce0-9e33-31b9a01f2e79 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:13.766769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8429764b-147b-4dc0-bdc8-26f7cf4dee5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:13.767381 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429764b-147b-4dc0-bdc8-26f7cf4dee5f None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:13.767672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429764b-147b-4dc0-bdc8-26f7cf4dee5f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:13.767953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429764b-147b-4dc0-bdc8-26f7cf4dee5f None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:13.768710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8429764b-147b-4dc0-bdc8-26f7cf4dee5f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:13.769019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8429764b-147b-4dc0-bdc8-26f7cf4dee5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:13.770435 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8429764b-147b-4dc0-bdc8-26f7cf4dee5f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:13.783568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 58/123] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:13.784422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8429764b-147b-4dc0-bdc8-26f7cf4dee5f None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:13.849586 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317dc05-c93e-4ce0-9e33-31b9a01f2e79 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:13.883301 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8429764b-147b-4dc0-bdc8-26f7cf4dee5f None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:13.884178 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140233c2-d768-40d8-b24a-f129f5f51a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:13.884843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140233c2-d768-40d8-b24a-f129f5f51a9c None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:13.885134 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140233c2-d768-40d8-b24a-f129f5f51a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:13.885400 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140233c2-d768-40d8-b24a-f129f5f51a9c None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:13.886156 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140233c2-d768-40d8-b24a-f129f5f51a9c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:13.887627 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140233c2-d768-40d8-b24a-f129f5f51a9c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:13.917967 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6ef78-7237-4260-a7e1-35ef8a5efef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:13.918810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6ef78-7237-4260-a7e1-35ef8a5efef8 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:13.919100 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6ef78-7237-4260-a7e1-35ef8a5efef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:13.919380 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6ef78-7237-4260-a7e1-35ef8a5efef8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:13.929219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6ef78-7237-4260-a7e1-35ef8a5efef8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:13.937185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-b6b6ef78-7237-4260-a7e1-35ef8a5efef8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:13.937185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:13.937185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:13.937185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:13.937185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:13.937185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:13.937185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:13.937185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:13.937185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:13.937185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:13.937185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:13.937185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:13.937185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:13.937185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:13.937185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:13.937185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:13.938317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:13.938317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:13.938317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:13.938317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:13.938317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:13.938317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:13.938317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:13.938317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:13.938317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:13.938317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:13.938317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:13.938317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:13.938317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:13.938317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:13.938317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:13.938317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:13.939667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:13.939667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:13.939667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:13.939667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:13.939667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:13.939667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:13.939667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:13.939667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:13.939667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:13.939667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:13.939667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:13.939667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:13.939667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 59/124] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:57:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:13.939667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6ef78-7237-4260-a7e1-35ef8a5efef8 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:13.986519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 66/125] 10.222.0.65 () {60 vars in 1141 bytes} [Tue May 10 19:57:13 2022] POST /identity/v3/projects => generated 302 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:57:13.987480 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140233c2-d768-40d8-b24a-f129f5f51a9c None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:14.075364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6ef78-7237-4260-a7e1-35ef8a5efef8 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:14.115737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140233c2-d768-40d8-b24a-f129f5f51a9c None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:14.121922 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ba4fd-5224-4c46-bf57-b1ee48ac6d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:14.122842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ba4fd-5224-4c46-bf57-b1ee48ac6d5b None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:14.123125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ba4fd-5224-4c46-bf57-b1ee48ac6d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:14.123401 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ba4fd-5224-4c46-bf57-b1ee48ac6d5b None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:14.125281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ba4fd-5224-4c46-bf57-b1ee48ac6d5b None admin] RBAC: Authorizing `identity:get_project(project_id=675b1b490743424e9d44dd2e484dfa95)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:14.127262 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ba4fd-5224-4c46-bf57-b1ee48ac6d5b None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:14.129797 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 60/126] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:13 2022] GET /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:14.135079 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ba4fd-5224-4c46-bf57-b1ee48ac6d5b None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:14.163511 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04da77f-2923-4b8f-9cf0-e1e6fd12ca1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:14.164662 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04da77f-2923-4b8f-9cf0-e1e6fd12ca1c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:14.164953 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04da77f-2923-4b8f-9cf0-e1e6fd12ca1c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:14.165405 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04da77f-2923-4b8f-9cf0-e1e6fd12ca1c None admin] PATH_INFO: `/v3/projects/9c647a719b03434cbdff743293674518` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:14.167672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04da77f-2923-4b8f-9cf0-e1e6fd12ca1c None admin] RBAC: Authorizing `identity:get_project(project_id=9c647a719b03434cbdff743293674518)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:14.169632 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04da77f-2923-4b8f-9cf0-e1e6fd12ca1c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:14.171546 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 67/127] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:13 2022] GET /identity/v3/projects/9c647a719b03434cbdff743293674518 => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:14.172380 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04da77f-2923-4b8f-9cf0-e1e6fd12ca1c None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:14.245328 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ba4fd-5224-4c46-bf57-b1ee48ac6d5b None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:14.284293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4a8b9-ff81-4acd-8dc1-58df4958082e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:14.285165 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4a8b9-ff81-4acd-8dc1-58df4958082e None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:14.285451 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4a8b9-ff81-4acd-8dc1-58df4958082e None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:14.285729 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4a8b9-ff81-4acd-8dc1-58df4958082e None admin] PATH_INFO: `/v3/users/bb370646880f4ad2b7b216d60a97e638` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:14.290107 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04da77f-2923-4b8f-9cf0-e1e6fd12ca1c None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:14.305182 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4a8b9-ff81-4acd-8dc1-58df4958082e None admin] RBAC: Authorizing `identity:get_user(user_id=bb370646880f4ad2b7b216d60a97e638)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:14.306721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4a8b9-ff81-4acd-8dc1-58df4958082e None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:14.308762 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 61/128] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:14 2022] GET /identity/v3/users/bb370646880f4ad2b7b216d60a97e638 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:14.309506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4a8b9-ff81-4acd-8dc1-58df4958082e None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:14.327631 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756e58e-aad8-4105-b85e-8bd0e151dbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:14.328629 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756e58e-aad8-4105-b85e-8bd0e151dbae None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:14.329113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756e58e-aad8-4105-b85e-8bd0e151dbae None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:14.329601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756e58e-aad8-4105-b85e-8bd0e151dbae None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:14.330587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756e58e-aad8-4105-b85e-8bd0e151dbae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:14.331107 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756e58e-aad8-4105-b85e-8bd0e151dbae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:14.332686 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756e58e-aad8-4105-b85e-8bd0e151dbae None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:14.342234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 68/129] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:57:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:14.343338 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756e58e-aad8-4105-b85e-8bd0e151dbae None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:14.431545 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4a8b9-ff81-4acd-8dc1-58df4958082e None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:14.446818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756e58e-aad8-4105-b85e-8bd0e151dbae None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:14.470281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bbe07-bf4c-4afc-8900-836490ecdd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:14.471177 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bbe07-bf4c-4afc-8900-836490ecdd3d None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:14.471460 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bbe07-bf4c-4afc-8900-836490ecdd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:14.471791 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bbe07-bf4c-4afc-8900-836490ecdd3d None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95/users/729fa49ea5f943b2b09c35b636e34ee5/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:14.473850 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bbe07-bf4c-4afc-8900-836490ecdd3d None admin] RBAC: Authorizing `identity:create_grant(project_id=675b1b490743424e9d44dd2e484dfa95, user_id=729fa49ea5f943b2b09c35b636e34ee5, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:14.475340 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bbe07-bf4c-4afc-8900-836490ecdd3d None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:14.483698 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41d156-39fb-40c4-96ef-ef223579a41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:14.484679 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41d156-39fb-40c4-96ef-ef223579a41b None admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:14.485039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41d156-39fb-40c4-96ef-ef223579a41b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:14.485391 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41d156-39fb-40c4-96ef-ef223579a41b None admin] PATH_INFO: `/v3/projects/9c647a719b03434cbdff743293674518/users/13e9677bc30f49ee88a08bd637b062b2/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:14.487498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41d156-39fb-40c4-96ef-ef223579a41b None admin] RBAC: Authorizing `identity:create_grant(project_id=9c647a719b03434cbdff743293674518, user_id=13e9677bc30f49ee88a08bd637b062b2, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:14.489008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41d156-39fb-40c4-96ef-ef223579a41b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:14.579465 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 62/130] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:57:14 2022] PUT /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95/users/729fa49ea5f943b2b09c35b636e34ee5/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:57:14.579465 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bbe07-bf4c-4afc-8900-836490ecdd3d None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:14.604330 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 69/131] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:57:14 2022] PUT /identity/v3/projects/9c647a719b03434cbdff743293674518/users/13e9677bc30f49ee88a08bd637b062b2/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:57:14.605300 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41d156-39fb-40c4-96ef-ef223579a41b None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:14.795350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bbe07-bf4c-4afc-8900-836490ecdd3d None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:14.801127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41d156-39fb-40c4-96ef-ef223579a41b None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:14.837385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93fffc-b87e-431f-a25d-990fcc96dbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:14.837966 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93fffc-b87e-431f-a25d-990fcc96dbae None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:14.838282 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93fffc-b87e-431f-a25d-990fcc96dbae None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:14.838563 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93fffc-b87e-431f-a25d-990fcc96dbae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:14.850969 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c38fc-2e65-437a-84ef-34dc8afc7f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:14.851879 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c38fc-2e65-437a-84ef-34dc8afc7f5c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:14.852168 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c38fc-2e65-437a-84ef-34dc8afc7f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:14.852449 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c38fc-2e65-437a-84ef-34dc8afc7f5c None admin] PATH_INFO: `/v3/users/13e9677bc30f49ee88a08bd637b062b2` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:14.853850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c38fc-2e65-437a-84ef-34dc8afc7f5c None admin] RBAC: Authorizing `identity:get_user(user_id=13e9677bc30f49ee88a08bd637b062b2)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:14.858122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c38fc-2e65-437a-84ef-34dc8afc7f5c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:14.862386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 70/132] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:14 2022] GET /identity/v3/users/13e9677bc30f49ee88a08bd637b062b2 => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:14.869431 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c38fc-2e65-437a-84ef-34dc8afc7f5c None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:14.966602 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93fffc-b87e-431f-a25d-990fcc96dbae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:14.968165 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93fffc-b87e-431f-a25d-990fcc96dbae None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:15.055425 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c38fc-2e65-437a-84ef-34dc8afc7f5c None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:15.120254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe6cc9-6860-4e4f-88ad-e2f21985aedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:15.121497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe6cc9-6860-4e4f-88ad-e2f21985aedd None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:15.122014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe6cc9-6860-4e4f-88ad-e2f21985aedd None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:15.122536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe6cc9-6860-4e4f-88ad-e2f21985aedd None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:15.124088 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe6cc9-6860-4e4f-88ad-e2f21985aedd None admin] RBAC: Authorizing `identity:get_project(project_id=675b1b490743424e9d44dd2e484dfa95)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:15.125850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe6cc9-6860-4e4f-88ad-e2f21985aedd None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:15.128260 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 71/133] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:14 2022] GET /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:15.135393 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe6cc9-6860-4e4f-88ad-e2f21985aedd None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:15.155446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 63/134] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:15.164415 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93fffc-b87e-431f-a25d-990fcc96dbae None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:15.300279 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93fffc-b87e-431f-a25d-990fcc96dbae None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:15.325603 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe6cc9-6860-4e4f-88ad-e2f21985aedd None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:15.342629 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7775b9-cba8-433a-af2d-6760160f5863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:15.343555 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7775b9-cba8-433a-af2d-6760160f5863 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:15.343845 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7775b9-cba8-433a-af2d-6760160f5863 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:15.344123 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7775b9-cba8-433a-af2d-6760160f5863 None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:15.345660 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7775b9-cba8-433a-af2d-6760160f5863 None admin] RBAC: Authorizing `identity:get_project(project_id=675b1b490743424e9d44dd2e484dfa95)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:15.347150 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7775b9-cba8-433a-af2d-6760160f5863 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:15.349642 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 64/135] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:15 2022] GET /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:15.355915 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7775b9-cba8-433a-af2d-6760160f5863 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:15.367545 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9140696-dee7-4905-82d5-a961338fe014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:15.369013 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9140696-dee7-4905-82d5-a961338fe014 None admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:15.369504 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9140696-dee7-4905-82d5-a961338fe014 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:15.369999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9140696-dee7-4905-82d5-a961338fe014 None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95/users/13e9677bc30f49ee88a08bd637b062b2/roles/4fbb0a346f5f4f36bb32664ee799db42` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:15.372549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9140696-dee7-4905-82d5-a961338fe014 None admin] RBAC: Authorizing `identity:create_grant(project_id=675b1b490743424e9d44dd2e484dfa95, user_id=13e9677bc30f49ee88a08bd637b062b2, role_id=4fbb0a346f5f4f36bb32664ee799db42)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:15.374257 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9140696-dee7-4905-82d5-a961338fe014 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:15.462745 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7775b9-cba8-433a-af2d-6760160f5863 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:15.469579 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 72/136] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:57:15 2022] PUT /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95/users/13e9677bc30f49ee88a08bd637b062b2/roles/4fbb0a346f5f4f36bb32664ee799db42 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:57:15.471404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0caf2-cc33-4349-b28e-a16984a9e608 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:15.471908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0caf2-cc33-4349-b28e-a16984a9e608 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:15.472394 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0caf2-cc33-4349-b28e-a16984a9e608 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:15.477699 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 73/137] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:15.480184 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab052c-a7f7-4c8c-a23d-f24f8a2d25aa None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:15.480716 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab052c-a7f7-4c8c-a23d-f24f8a2d25aa None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:15.481201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab052c-a7f7-4c8c-a23d-f24f8a2d25aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:15.499712 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941bc48-02e4-4b79-9282-1ebdff0b4f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:15.500450 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941bc48-02e4-4b79-9282-1ebdff0b4f65 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:15.500736 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941bc48-02e4-4b79-9282-1ebdff0b4f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:15.501014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941bc48-02e4-4b79-9282-1ebdff0b4f65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:15.502059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941bc48-02e4-4b79-9282-1ebdff0b4f65 None admin] RBAC: Adding query filter params (role.id=f7a4054724ca4163960cf635d13b055f, scope.project.id=675b1b490743424e9d44dd2e484dfa95, user.id=bb370646880f4ad2b7b216d60a97e638) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:15.506338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941bc48-02e4-4b79-9282-1ebdff0b4f65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:15.506338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941bc48-02e4-4b79-9282-1ebdff0b4f65 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:15.511958 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 65/138] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:57:15 2022] GET /identity/v3/role_assignments?user.id=bb370646880f4ad2b7b216d60a97e638&scope.project.id=675b1b490743424e9d44dd2e484dfa95&role.id=f7a4054724ca4163960cf635d13b055f => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:15.515275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-41ab052c-a7f7-4c8c-a23d-f24f8a2d25aa None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:15.516895 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-41ab052c-a7f7-4c8c-a23d-f24f8a2d25aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:15.627940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 74/139] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:15.677583 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941bc48-02e4-4b79-9282-1ebdff0b4f65 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:15.798757 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941bc48-02e4-4b79-9282-1ebdff0b4f65 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:15.834430 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3c544-1c32-4c5d-98e1-104f9034683d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:15.835851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3c544-1c32-4c5d-98e1-104f9034683d None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:15.836137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3c544-1c32-4c5d-98e1-104f9034683d None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:15.836408 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3c544-1c32-4c5d-98e1-104f9034683d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:15.851534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3c544-1c32-4c5d-98e1-104f9034683d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:15.859311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-1ee3c544-1c32-4c5d-98e1-104f9034683d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:15.859311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:15.859311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:15.859311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:15.859311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:15.859311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:15.859311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:15.859311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:15.859311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:15.859311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:15.859311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:15.859311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:15.859311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:15.859311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:15.859311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:15.859311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:15.860163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:15.860163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:15.860163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:15.860163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:15.860163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:15.860163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:15.860163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:15.860163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:15.860163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:15.860163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:15.860163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:15.860163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:15.860163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:15.860163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:15.860163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:15.860163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:15.860924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:15.860924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:15.860924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:15.860924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:15.860924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:15.860924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:15.860924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:15.860924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:15.860924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:15.860924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:15.860924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:15.860924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:15.860924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 66/140] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:15.866256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab052c-a7f7-4c8c-a23d-f24f8a2d25aa None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:15.967308 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab052c-a7f7-4c8c-a23d-f24f8a2d25aa None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:16.001350 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805daa1-b4a3-4f07-9a59-c6ff6d6e1d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:16.002028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805daa1-b4a3-4f07-9a59-c6ff6d6e1d48 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:16.002346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805daa1-b4a3-4f07-9a59-c6ff6d6e1d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:16.002632 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805daa1-b4a3-4f07-9a59-c6ff6d6e1d48 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:16.003394 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805daa1-b4a3-4f07-9a59-c6ff6d6e1d48 None admin] RBAC: Adding query filter params (name=member) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:16.003708 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805daa1-b4a3-4f07-9a59-c6ff6d6e1d48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:16.005086 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805daa1-b4a3-4f07-9a59-c6ff6d6e1d48 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:16.014230 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 75/141] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:16.019502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3c544-1c32-4c5d-98e1-104f9034683d None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:16.135051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3c544-1c32-4c5d-98e1-104f9034683d None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:16.211635 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c35968-b7ac-404e-ba95-d3766c2767a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:16.212532 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c35968-b7ac-404e-ba95-d3766c2767a0 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:16.212817 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c35968-b7ac-404e-ba95-d3766c2767a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:16.213096 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c35968-b7ac-404e-ba95-d3766c2767a0 None admin] PATH_INFO: `/v3/users/13e9677bc30f49ee88a08bd637b062b2` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:16.214923 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c35968-b7ac-404e-ba95-d3766c2767a0 None admin] RBAC: Authorizing `identity:get_user(user_id=13e9677bc30f49ee88a08bd637b062b2)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:16.216454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c35968-b7ac-404e-ba95-d3766c2767a0 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:16.218318 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 67/142] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:16 2022] GET /identity/v3/users/13e9677bc30f49ee88a08bd637b062b2 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:16.223593 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805daa1-b4a3-4f07-9a59-c6ff6d6e1d48 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:16.453681 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805daa1-b4a3-4f07-9a59-c6ff6d6e1d48 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:16.502866 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bdf46-ccef-402e-b607-a2950ca07af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:16.503861 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bdf46-ccef-402e-b607-a2950ca07af5 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:16.504363 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bdf46-ccef-402e-b607-a2950ca07af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:16.504838 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bdf46-ccef-402e-b607-a2950ca07af5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:16.596327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874bdf46-ccef-402e-b607-a2950ca07af5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:16.602154 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874bdf46-ccef-402e-b607-a2950ca07af5 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:16.728745 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 76/143] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:16.738665 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c35968-b7ac-404e-ba95-d3766c2767a0 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:16.909845 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c35968-b7ac-404e-ba95-d3766c2767a0 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:16.954809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97119504-48a3-47d1-824f-6639e6ef8931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:16.955691 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97119504-48a3-47d1-824f-6639e6ef8931 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:16.955981 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97119504-48a3-47d1-824f-6639e6ef8931 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:16.956264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97119504-48a3-47d1-824f-6639e6ef8931 None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:16.957508 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97119504-48a3-47d1-824f-6639e6ef8931 None admin] RBAC: Authorizing `identity:get_project(project_id=675b1b490743424e9d44dd2e484dfa95)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:16.958992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97119504-48a3-47d1-824f-6639e6ef8931 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:16.961225 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 68/144] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:16 2022] GET /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:16.969042 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bdf46-ccef-402e-b607-a2950ca07af5 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:17.083902 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c16ad5-71d4-468e-9fb0-0e11375229e7 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:17.084241 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c16ad5-71d4-468e-9fb0-0e11375229e7 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:17.084483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c16ad5-71d4-468e-9fb0-0e11375229e7 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:17.086488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 69/145] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:17.089286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bdf46-ccef-402e-b607-a2950ca07af5 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:17.095536 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a34784-1438-429b-bf8c-911919913d9f None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:17.095835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a34784-1438-429b-bf8c-911919913d9f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:17.096127 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a34784-1438-429b-bf8c-911919913d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:17.125077 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-17a34784-1438-429b-bf8c-911919913d9f None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:17.127116 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-17a34784-1438-429b-bf8c-911919913d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:17.133321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c4cca-3a67-45c1-9ea8-01f0e258e4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:17.134308 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c4cca-3a67-45c1-9ea8-01f0e258e4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:17.134794 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c4cca-3a67-45c1-9ea8-01f0e258e4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:17.135289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c4cca-3a67-45c1-9ea8-01f0e258e4d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:17.136338 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c4cca-3a67-45c1-9ea8-01f0e258e4d0 None admin] RBAC: Adding query filter params (role.id=5f62ce25636443f8a5add67f208afe62, scope.project.id=675b1b490743424e9d44dd2e484dfa95, user.id=13e9677bc30f49ee88a08bd637b062b2) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:17.138875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c4cca-3a67-45c1-9ea8-01f0e258e4d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:17.140544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c4cca-3a67-45c1-9ea8-01f0e258e4d0 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:17.152763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 77/146] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:57:16 2022] GET /identity/v3/role_assignments?user.id=13e9677bc30f49ee88a08bd637b062b2&scope.project.id=675b1b490743424e9d44dd2e484dfa95&role.id=5f62ce25636443f8a5add67f208afe62 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:17.228781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 70/147] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:17.270410 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c4cca-3a67-45c1-9ea8-01f0e258e4d0 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:17.384426 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c4cca-3a67-45c1-9ea8-01f0e258e4d0 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:17.436669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d402c8b-7d62-4a5d-99a3-0260254f6cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:17.437656 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d402c8b-7d62-4a5d-99a3-0260254f6cec None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:17.438136 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d402c8b-7d62-4a5d-99a3-0260254f6cec None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:17.438589 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d402c8b-7d62-4a5d-99a3-0260254f6cec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:17.440062 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d402c8b-7d62-4a5d-99a3-0260254f6cec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:17.441658 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d402c8b-7d62-4a5d-99a3-0260254f6cec None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:17.443815 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 78/148] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:57:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:17.450200 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a34784-1438-429b-bf8c-911919913d9f None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:17.585551 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a34784-1438-429b-bf8c-911919913d9f None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:17.628013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b0f26-f796-4008-aa04-b6dc41506ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:17.628695 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b0f26-f796-4008-aa04-b6dc41506ebb None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:17.628982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b0f26-f796-4008-aa04-b6dc41506ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:17.629259 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b0f26-f796-4008-aa04-b6dc41506ebb None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:17.630002 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b0f26-f796-4008-aa04-b6dc41506ebb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:17.632136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b0f26-f796-4008-aa04-b6dc41506ebb None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:17.641280 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1a5ab-6941-48c7-9485-b033a672a831 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:17.641933 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1a5ab-6941-48c7-9485-b033a672a831 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:17.642435 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1a5ab-6941-48c7-9485-b033a672a831 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:17.644419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 79/149] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:57:17.650509 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062efa3-c09e-4df1-96dd-49b3173540ba None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:17.651083 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062efa3-c09e-4df1-96dd-49b3173540ba None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:17.651623 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062efa3-c09e-4df1-96dd-49b3173540ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:17.672664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-3062efa3-c09e-4df1-96dd-49b3173540ba None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:17.674263 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3062efa3-c09e-4df1-96dd-49b3173540ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:17.751738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 80/150] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:17.763192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 71/151] 10.222.0.65 () {60 vars in 1132 bytes} [Tue May 10 19:57:17 2022] POST /identity/v3/users => generated 283 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:57:17.778636 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062efa3-c09e-4df1-96dd-49b3173540ba None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:17.905654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062efa3-c09e-4df1-96dd-49b3173540ba None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:17.947752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e36776-c462-4726-a274-df069c707fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:17.948563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e36776-c462-4726-a274-df069c707fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:17.948853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e36776-c462-4726-a274-df069c707fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:17.949133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e36776-c462-4726-a274-df069c707fa7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:17.967118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e36776-c462-4726-a274-df069c707fa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:17.974863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-11e36776-c462-4726-a274-df069c707fa7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:17.974863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:17.974863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:17.974863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:17.974863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:17.974863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:17.974863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:17.974863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:17.974863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:17.974863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:17.974863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:17.974863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:17.974863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:17.974863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:17.974863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:17.974863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:17.976340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:17.976340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:17.976340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:17.976340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:17.976340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:17.976340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:17.976340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:17.976340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:17.976340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:17.976340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:17.976340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:17.976340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:17.976340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:17.976340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:17.976340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:17.976340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:17.977197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:17.977197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:17.977197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:17.977197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:17.977197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:17.977197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:17.977197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:17.977197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:17.977197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:17.977197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:17.977197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:17.977197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:17.977197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 81/152] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:57:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:17.982495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b0f26-f796-4008-aa04-b6dc41506ebb None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:18.131191 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b0f26-f796-4008-aa04-b6dc41506ebb None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:18.170820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d6ca8-74ef-4136-bf03-6088827b50f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:18.171428 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d6ca8-74ef-4136-bf03-6088827b50f9 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:18.171745 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d6ca8-74ef-4136-bf03-6088827b50f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:18.171989 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d6ca8-74ef-4136-bf03-6088827b50f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:18.172750 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d6ca8-74ef-4136-bf03-6088827b50f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:18.173059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d6ca8-74ef-4136-bf03-6088827b50f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:18.174450 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d6ca8-74ef-4136-bf03-6088827b50f9 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:18.185838 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 72/153] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:18.191196 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e36776-c462-4726-a274-df069c707fa7 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:18.261387 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0e902-171d-4844-bff8-464dd0e86a26 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:18.261697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0e902-171d-4844-bff8-464dd0e86a26 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:18.261941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0e902-171d-4844-bff8-464dd0e86a26 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:18.263594 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 73/154] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:57:18.268782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f8ca3-8f34-4e58-b8b4-a6e9f515c355 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:18.269076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f8ca3-8f34-4e58-b8b4-a6e9f515c355 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:18.269356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f8ca3-8f34-4e58-b8b4-a6e9f515c355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:18.286858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-202f8ca3-8f34-4e58-b8b4-a6e9f515c355 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:18.291447 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-202f8ca3-8f34-4e58-b8b4-a6e9f515c355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:18.317697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e36776-c462-4726-a274-df069c707fa7 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:18.358640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53fecd-98b7-4493-9c68-8aeaeeff1656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:18.359521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53fecd-98b7-4493-9c68-8aeaeeff1656 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:18.359800 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53fecd-98b7-4493-9c68-8aeaeeff1656 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:18.360073 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53fecd-98b7-4493-9c68-8aeaeeff1656 None admin] PATH_INFO: `/v3/users/729fa49ea5f943b2b09c35b636e34ee5` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:18.362032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 74/155] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:18.362746 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53fecd-98b7-4493-9c68-8aeaeeff1656 None admin] RBAC: Authorizing `identity:get_user(user_id=729fa49ea5f943b2b09c35b636e34ee5)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:18.364196 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53fecd-98b7-4493-9c68-8aeaeeff1656 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:18.366464 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 82/156] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:18 2022] GET /identity/v3/users/729fa49ea5f943b2b09c35b636e34ee5 => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:18.372920 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f8ca3-8f34-4e58-b8b4-a6e9f515c355 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:18.392455 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53fecd-98b7-4493-9c68-8aeaeeff1656 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:18.495484 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f8ca3-8f34-4e58-b8b4-a6e9f515c355 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:18.510342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53fecd-98b7-4493-9c68-8aeaeeff1656 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:18.542015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d537e2-ef01-404b-b81f-39854d0d34b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:18.542745 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d537e2-ef01-404b-b81f-39854d0d34b1 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:18.543038 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d537e2-ef01-404b-b81f-39854d0d34b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:18.543320 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d537e2-ef01-404b-b81f-39854d0d34b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:18.552295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3869d5-bb6f-4054-bc9a-83ef0afc0c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:18.553344 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3869d5-bb6f-4054-bc9a-83ef0afc0c08 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:18.553762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3869d5-bb6f-4054-bc9a-83ef0afc0c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:18.554197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3869d5-bb6f-4054-bc9a-83ef0afc0c08 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:18.580329 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3869d5-bb6f-4054-bc9a-83ef0afc0c08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:18.587420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-ef3869d5-bb6f-4054-bc9a-83ef0afc0c08 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:18.587420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:18.587420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:18.587420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:18.587420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:18.587420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:18.587420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:18.587420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:18.587420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:18.587420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:18.587420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:18.587420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:18.587420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:18.587420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:18.587420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:18.587420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:18.588261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:18.588261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:18.588261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:18.588261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:18.588261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:18.588261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:18.588261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:18.588261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:18.588261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:18.588261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:18.588261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:18.588261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:18.588261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:18.588261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:18.588261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:18.588261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:18.589063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:18.589063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:18.589063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:18.589063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:18.589063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:18.589063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:18.589063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:18.589063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:18.589063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:18.589063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:18.589063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:18.589063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:18.589063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 83/157] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:18.602724 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3869d5-bb6f-4054-bc9a-83ef0afc0c08 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:18.634281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d537e2-ef01-404b-b81f-39854d0d34b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:18.636048 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d537e2-ef01-404b-b81f-39854d0d34b1 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:18.713160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3869d5-bb6f-4054-bc9a-83ef0afc0c08 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:18.751527 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb124216-fd79-48e3-a6e6-be4f32f48db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:18.752547 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb124216-fd79-48e3-a6e6-be4f32f48db4 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:18.753024 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb124216-fd79-48e3-a6e6-be4f32f48db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:18.753489 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb124216-fd79-48e3-a6e6-be4f32f48db4 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:18.754494 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb124216-fd79-48e3-a6e6-be4f32f48db4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:18.755015 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb124216-fd79-48e3-a6e6-be4f32f48db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:18.756621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb124216-fd79-48e3-a6e6-be4f32f48db4 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:18.756854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 75/158] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 19:57:18.762676 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d537e2-ef01-404b-b81f-39854d0d34b1 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:18.766431 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 84/159] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:18.777332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb124216-fd79-48e3-a6e6-be4f32f48db4 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:18.883087 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d537e2-ef01-404b-b81f-39854d0d34b1 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:18.897888 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb124216-fd79-48e3-a6e6-be4f32f48db4 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:18.931937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c4554-d5b3-4eaa-abe5-605051711e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:18.932899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c4554-d5b3-4eaa-abe5-605051711e0b None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:18.933190 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c4554-d5b3-4eaa-abe5-605051711e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:18.933483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c4554-d5b3-4eaa-abe5-605051711e0b None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:18.935455 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c4554-d5b3-4eaa-abe5-605051711e0b None admin] RBAC: Authorizing `identity:get_project(project_id=675b1b490743424e9d44dd2e484dfa95)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:18.936906 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c4554-d5b3-4eaa-abe5-605051711e0b None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:18.938651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 76/160] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:18 2022] GET /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:18.948457 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c4554-d5b3-4eaa-abe5-605051711e0b None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:18.949995 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf98a36-26d4-4b05-8bdd-5316d078edcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:18.950940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf98a36-26d4-4b05-8bdd-5316d078edcb None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:18.951226 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf98a36-26d4-4b05-8bdd-5316d078edcb None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:18.951502 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf98a36-26d4-4b05-8bdd-5316d078edcb None admin] PATH_INFO: `/v3/users/13e9677bc30f49ee88a08bd637b062b2` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:18.953225 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf98a36-26d4-4b05-8bdd-5316d078edcb None admin] RBAC: Authorizing `identity:get_user(user_id=13e9677bc30f49ee88a08bd637b062b2)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:18.954728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf98a36-26d4-4b05-8bdd-5316d078edcb None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:18.960959 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 85/161] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:18 2022] GET /identity/v3/users/13e9677bc30f49ee88a08bd637b062b2 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:18.966544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf98a36-26d4-4b05-8bdd-5316d078edcb None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:19.080185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c4554-d5b3-4eaa-abe5-605051711e0b None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:19.127555 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb932a-fd83-404c-b610-a5611d275df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:19.128645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb932a-fd83-404c-b610-a5611d275df4 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:19.129121 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb932a-fd83-404c-b610-a5611d275df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:19.129231 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf98a36-26d4-4b05-8bdd-5316d078edcb None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:19.130018 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb932a-fd83-404c-b610-a5611d275df4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:19.132534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb932a-fd83-404c-b610-a5611d275df4 None admin] RBAC: Adding query filter params (role.id=5c87e804d0764309ba08ad6e06b23a87, scope.project.id=675b1b490743424e9d44dd2e484dfa95, user.id=729fa49ea5f943b2b09c35b636e34ee5) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:19.133089 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb932a-fd83-404c-b610-a5611d275df4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:19.134770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb932a-fd83-404c-b610-a5611d275df4 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:19.146071 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 77/162] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:57:18 2022] GET /identity/v3/role_assignments?user.id=729fa49ea5f943b2b09c35b636e34ee5&scope.project.id=675b1b490743424e9d44dd2e484dfa95&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:19.148054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d15108-596a-4521-89ce-41ef5794c98a None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:19.148346 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d15108-596a-4521-89ce-41ef5794c98a None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:19.148620 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d15108-596a-4521-89ce-41ef5794c98a None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:19.151446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 78/163] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:19.157861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1500fd-6594-4400-aa25-6bcde15bd599 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:19.158175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1500fd-6594-4400-aa25-6bcde15bd599 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:19.158445 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1500fd-6594-4400-aa25-6bcde15bd599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:19.189125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-dd1500fd-6594-4400-aa25-6bcde15bd599 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:19.191055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-dd1500fd-6594-4400-aa25-6bcde15bd599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:19.222839 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0355c-fff3-4444-8584-9b6298e05150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:19.223253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0355c-fff3-4444-8584-9b6298e05150 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:19.223541 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0355c-fff3-4444-8584-9b6298e05150 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:19.223817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0355c-fff3-4444-8584-9b6298e05150 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:19.292702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 79/164] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:19.295815 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341893a-8887-4c49-a128-6a949c4e6d1c None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:19.295815 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341893a-8887-4c49-a128-6a949c4e6d1c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:19.295815 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341893a-8887-4c49-a128-6a949c4e6d1c None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:19.296678 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 80/165] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:19.301930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cfd03-a906-48b4-9b5c-7a4b01b303a1 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:19.302247 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cfd03-a906-48b4-9b5c-7a4b01b303a1 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:19.302574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cfd03-a906-48b4-9b5c-7a4b01b303a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:19.317778 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0355c-fff3-4444-8584-9b6298e05150 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:19.321349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0355c-fff3-4444-8584-9b6298e05150 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:19.338356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-006cfd03-a906-48b4-9b5c-7a4b01b303a1 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:19.339606 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-006cfd03-a906-48b4-9b5c-7a4b01b303a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:19.441491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 86/166] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:19.446941 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0355c-fff3-4444-8584-9b6298e05150 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:19.476049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 81/167] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:57:19.476636 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006cfd03-a906-48b4-9b5c-7a4b01b303a1 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:19.560289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0355c-fff3-4444-8584-9b6298e05150 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:19.601897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04a856-dc73-45b2-abca-caf7d343f46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:19.602742 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04a856-dc73-45b2-abca-caf7d343f46e None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:19.603021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04a856-dc73-45b2-abca-caf7d343f46e None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:19.603291 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04a856-dc73-45b2-abca-caf7d343f46e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:19.627536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04a856-dc73-45b2-abca-caf7d343f46e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:19.644265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-5f04a856-dc73-45b2-abca-caf7d343f46e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:19.644265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:19.644265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:19.644265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:19.644265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:19.644265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:19.644265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:19.644265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:19.644265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:19.644265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:19.644265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:19.644265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:19.644265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:19.644265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:19.644265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:19.644265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:19.645207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:19.645207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:19.645207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:19.645207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:19.645207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:19.645207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:19.645207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:19.645207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:19.645207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:19.645207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:19.645207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:19.645207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:19.645207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:19.645207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:19.645207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:19.645207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:19.645989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:19.645989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:19.645989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:19.645989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:19.645989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:19.645989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:19.645989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:19.645989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:19.645989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:19.645989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:19.645989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:19.645989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:19.649019 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 87/168] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:57:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:19.650155 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04a856-dc73-45b2-abca-caf7d343f46e None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:19.653059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006cfd03-a906-48b4-9b5c-7a4b01b303a1 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:19.706639 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45e1e3-1880-488c-b9da-44fc4393a424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:19.707503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45e1e3-1880-488c-b9da-44fc4393a424 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:19.707777 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45e1e3-1880-488c-b9da-44fc4393a424 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:19.708042 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45e1e3-1880-488c-b9da-44fc4393a424 None admin] PATH_INFO: `/v3/projects/9c647a719b03434cbdff743293674518` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:19.709347 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45e1e3-1880-488c-b9da-44fc4393a424 None admin] RBAC: Authorizing `identity:get_project(project_id=9c647a719b03434cbdff743293674518)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:19.712908 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45e1e3-1880-488c-b9da-44fc4393a424 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:19.714654 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 82/169] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:19 2022] GET /identity/v3/projects/9c647a719b03434cbdff743293674518 => generated 312 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:19.715425 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45e1e3-1880-488c-b9da-44fc4393a424 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:19.796611 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04a856-dc73-45b2-abca-caf7d343f46e None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:19.846848 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2fc4a-7c44-468c-9456-215a12780c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:19.847661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2fc4a-7c44-468c-9456-215a12780c08 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:19.847955 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2fc4a-7c44-468c-9456-215a12780c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:19.848233 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2fc4a-7c44-468c-9456-215a12780c08 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:19.865166 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45e1e3-1880-488c-b9da-44fc4393a424 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:19.878672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2fc4a-7c44-468c-9456-215a12780c08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:19.886945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-d7c2fc4a-7c44-468c-9456-215a12780c08 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 19:57:19.886945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:19.886945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:19.886945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:19.886945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:19.886945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:19.886945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:19.886945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:19.886945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:19.886945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:19.886945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:19.886945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:19.886945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:19.886945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:19.886945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:19.886945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:19.887837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:19.887837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:19.887837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:19.887837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:19.887837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:19.887837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:19.887837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:19.887837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:19.887837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:19.887837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:19.887837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:19.887837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:19.887837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:19.887837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:19.887837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:19.887837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:19.888622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:19.888622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:19.888622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:19.888622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:19.888622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:19.888622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:19.888622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:19.888622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:19.888622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:19.888622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:19.888622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 19:57:19.888622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:19.894507 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 88/170] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:19.895361 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2fc4a-7c44-468c-9456-215a12780c08 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:19.903908 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b59d52-af85-4da3-9313-fd5faf8d4e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:19.904524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b59d52-af85-4da3-9313-fd5faf8d4e97 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:19.906112 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b59d52-af85-4da3-9313-fd5faf8d4e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:19.906112 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b59d52-af85-4da3-9313-fd5faf8d4e97 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:19.906112 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b59d52-af85-4da3-9313-fd5faf8d4e97 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:19.906112 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b59d52-af85-4da3-9313-fd5faf8d4e97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:19.907474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b59d52-af85-4da3-9313-fd5faf8d4e97 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:19.923944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 83/171] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:57:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:19.924774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b59d52-af85-4da3-9313-fd5faf8d4e97 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:20.057623 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b59d52-af85-4da3-9313-fd5faf8d4e97 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:20.093531 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122cf23e-1369-47cb-b0dc-f3a0a6146afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:20.094247 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cf23e-1369-47cb-b0dc-f3a0a6146afe None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:20.094532 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cf23e-1369-47cb-b0dc-f3a0a6146afe None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:20.094851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cf23e-1369-47cb-b0dc-f3a0a6146afe None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:20.095629 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122cf23e-1369-47cb-b0dc-f3a0a6146afe None admin] RBAC: Adding query filter params (name=reader) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:20.095932 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122cf23e-1369-47cb-b0dc-f3a0a6146afe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:20.097321 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122cf23e-1369-47cb-b0dc-f3a0a6146afe None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:20.106524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 84/172] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:20.107312 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122cf23e-1369-47cb-b0dc-f3a0a6146afe None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:20.109469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2fc4a-7c44-468c-9456-215a12780c08 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:20.170603 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5ff19-7f07-4b7a-82fd-bd24bcd73523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:20.171256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5ff19-7f07-4b7a-82fd-bd24bcd73523 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:20.171528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5ff19-7f07-4b7a-82fd-bd24bcd73523 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:20.171803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5ff19-7f07-4b7a-82fd-bd24bcd73523 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:20.172700 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff5ff19-7f07-4b7a-82fd-bd24bcd73523 None admin] RBAC: Adding query filter params (role.id=5f62ce25636443f8a5add67f208afe62, scope.project.id=9c647a719b03434cbdff743293674518, user.id=13e9677bc30f49ee88a08bd637b062b2) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:20.173007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff5ff19-7f07-4b7a-82fd-bd24bcd73523 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:20.175945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff5ff19-7f07-4b7a-82fd-bd24bcd73523 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:20.188474 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 89/173] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:57:19 2022] GET /identity/v3/role_assignments?user.id=13e9677bc30f49ee88a08bd637b062b2&scope.project.id=9c647a719b03434cbdff743293674518&role.id=5f62ce25636443f8a5add67f208afe62 => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:20.189309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5ff19-7f07-4b7a-82fd-bd24bcd73523 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:20.328995 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122cf23e-1369-47cb-b0dc-f3a0a6146afe None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:20.337660 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5ff19-7f07-4b7a-82fd-bd24bcd73523 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:20.376843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec39d1-5e7d-4144-9a42-1dad038f3aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:20.377720 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec39d1-5e7d-4144-9a42-1dad038f3aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:20.378001 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec39d1-5e7d-4144-9a42-1dad038f3aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:20.378425 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec39d1-5e7d-4144-9a42-1dad038f3aa8 None admin] PATH_INFO: `/v3/users/bb370646880f4ad2b7b216d60a97e638` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:20.380198 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec39d1-5e7d-4144-9a42-1dad038f3aa8 None admin] RBAC: Authorizing `identity:get_user(user_id=bb370646880f4ad2b7b216d60a97e638)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:20.383196 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec39d1-5e7d-4144-9a42-1dad038f3aa8 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:20.384732 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 90/174] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:20 2022] GET /identity/v3/users/bb370646880f4ad2b7b216d60a97e638 => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:20.403282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec39d1-5e7d-4144-9a42-1dad038f3aa8 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:20.423246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a021359-ea31-4000-bcf0-1094e7a637f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:20.424392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a021359-ea31-4000-bcf0-1094e7a637f0 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:20.425284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a021359-ea31-4000-bcf0-1094e7a637f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:20.425717 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a021359-ea31-4000-bcf0-1094e7a637f0 None admin] PATH_INFO: `/v3/users/13e9677bc30f49ee88a08bd637b062b2` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:20.427189 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a021359-ea31-4000-bcf0-1094e7a637f0 None admin] RBAC: Authorizing `identity:get_user(user_id=13e9677bc30f49ee88a08bd637b062b2)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:20.435057 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a021359-ea31-4000-bcf0-1094e7a637f0 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:20.436893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 85/175] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:20 2022] GET /identity/v3/users/13e9677bc30f49ee88a08bd637b062b2 => generated 275 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:20.442405 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a021359-ea31-4000-bcf0-1094e7a637f0 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:20.624039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec39d1-5e7d-4144-9a42-1dad038f3aa8 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:20.644875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a021359-ea31-4000-bcf0-1094e7a637f0 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:20.684085 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b659484-2589-4018-a5be-f226a0ce7f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:20.684951 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b659484-2589-4018-a5be-f226a0ce7f44 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:20.685237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b659484-2589-4018-a5be-f226a0ce7f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:20.685513 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b659484-2589-4018-a5be-f226a0ce7f44 None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:20.686872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b659484-2589-4018-a5be-f226a0ce7f44 None admin] RBAC: Authorizing `identity:get_project(project_id=675b1b490743424e9d44dd2e484dfa95)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:20.690158 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b659484-2589-4018-a5be-f226a0ce7f44 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:20.696364 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 91/176] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:20 2022] GET /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95 => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:20.699870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b659484-2589-4018-a5be-f226a0ce7f44 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:20.713544 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cabbc34-9225-45eb-99d7-d2b00349f676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:20.714256 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cabbc34-9225-45eb-99d7-d2b00349f676 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:20.714540 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cabbc34-9225-45eb-99d7-d2b00349f676 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:20.714819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cabbc34-9225-45eb-99d7-d2b00349f676 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:20.806509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cabbc34-9225-45eb-99d7-d2b00349f676 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:20.808434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cabbc34-9225-45eb-99d7-d2b00349f676 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:20.855313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b659484-2589-4018-a5be-f226a0ce7f44 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:20.904280 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853bf80-7b83-490f-b081-808bf0f007b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:20.905794 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853bf80-7b83-490f-b081-808bf0f007b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:20.906478 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853bf80-7b83-490f-b081-808bf0f007b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:20.907232 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853bf80-7b83-490f-b081-808bf0f007b8 None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95/users/bb370646880f4ad2b7b216d60a97e638/roles/f7a4054724ca4163960cf635d13b055f` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:20.912009 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853bf80-7b83-490f-b081-808bf0f007b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=675b1b490743424e9d44dd2e484dfa95, user_id=bb370646880f4ad2b7b216d60a97e638, role_id=f7a4054724ca4163960cf635d13b055f)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:20.918675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853bf80-7b83-490f-b081-808bf0f007b8 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:20.930324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 86/177] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 19:57:20.936069 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cabbc34-9225-45eb-99d7-d2b00349f676 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:20.998987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 92/178] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:57:20 2022] PUT /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95/users/bb370646880f4ad2b7b216d60a97e638/roles/f7a4054724ca4163960cf635d13b055f => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:57:21.051438 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cabbc34-9225-45eb-99d7-d2b00349f676 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:21.090358 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c3b93-b90e-4657-b6dc-8a9432405127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:21.091225 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3b93-b90e-4657-b6dc-8a9432405127 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:21.091503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3b93-b90e-4657-b6dc-8a9432405127 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:21.091770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3b93-b90e-4657-b6dc-8a9432405127 None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:21.093074 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c3b93-b90e-4657-b6dc-8a9432405127 None admin] RBAC: Authorizing `identity:get_project(project_id=675b1b490743424e9d44dd2e484dfa95)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:21.094661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c3b93-b90e-4657-b6dc-8a9432405127 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:21.096393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 87/179] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:20 2022] GET /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:21.103081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853bf80-7b83-490f-b081-808bf0f007b8 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:21.232687 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853bf80-7b83-490f-b081-808bf0f007b8 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:21.282165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2294ad-ded5-4be2-8de6-f2c447e3942d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:21.282826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2294ad-ded5-4be2-8de6-f2c447e3942d None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:21.283109 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2294ad-ded5-4be2-8de6-f2c447e3942d None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:21.283382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2294ad-ded5-4be2-8de6-f2c447e3942d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:21.284234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2294ad-ded5-4be2-8de6-f2c447e3942d None admin] RBAC: Adding query filter params (role.id=4fbb0a346f5f4f36bb32664ee799db42, scope.project.id=675b1b490743424e9d44dd2e484dfa95, user.id=13e9677bc30f49ee88a08bd637b062b2) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:21.284543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2294ad-ded5-4be2-8de6-f2c447e3942d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:21.285980 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2294ad-ded5-4be2-8de6-f2c447e3942d None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:21.293296 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 93/180] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:57:21 2022] GET /identity/v3/role_assignments?user.id=13e9677bc30f49ee88a08bd637b062b2&scope.project.id=675b1b490743424e9d44dd2e484dfa95&role.id=4fbb0a346f5f4f36bb32664ee799db42 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:21.630395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee972c0a-e90d-4d9e-a4b6-f3fdfc057685 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:21.630578 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee972c0a-e90d-4d9e-a4b6-f3fdfc057685 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:21.630858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee972c0a-e90d-4d9e-a4b6-f3fdfc057685 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:21.633193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 88/181] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:21.638722 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5083f37-313c-4129-9cb4-8a47f81662b1 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:21.639063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5083f37-313c-4129-9cb4-8a47f81662b1 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:21.639271 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5083f37-313c-4129-9cb4-8a47f81662b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:21.657197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-b5083f37-313c-4129-9cb4-8a47f81662b1 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:21.658411 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b5083f37-313c-4129-9cb4-8a47f81662b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:21.684896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e61e8a-61ca-4848-9200-d2daa049578a None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:21.685203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e61e8a-61ca-4848-9200-d2daa049578a None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:21.685446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e61e8a-61ca-4848-9200-d2daa049578a None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:21.687103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 89/182] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:21.692325 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3607c-43c8-48ab-b37d-55d835de9ae1 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:21.692614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3607c-43c8-48ab-b37d-55d835de9ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:21.692897 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3607c-43c8-48ab-b37d-55d835de9ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:21.718405 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-b4d3607c-43c8-48ab-b37d-55d835de9ae1 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:21.719593 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b4d3607c-43c8-48ab-b37d-55d835de9ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:21.740251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 94/183] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:21.765991 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5083f37-313c-4129-9cb4-8a47f81662b1 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:21.802158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 90/184] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:21.804348 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a0d9b-82e8-4b6c-ae08-e7eb62bf0d33 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:21.804808 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a0d9b-82e8-4b6c-ae08-e7eb62bf0d33 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:21.805230 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a0d9b-82e8-4b6c-ae08-e7eb62bf0d33 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:21.806604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 91/185] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:21.814690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d028a-47ba-4d93-981f-6903a82b2cb3 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:21.814974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d028a-47ba-4d93-981f-6903a82b2cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:21.815251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d028a-47ba-4d93-981f-6903a82b2cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:21.833706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-e47d028a-47ba-4d93-981f-6903a82b2cb3 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:21.835017 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-e47d028a-47ba-4d93-981f-6903a82b2cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:21.891528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5083f37-313c-4129-9cb4-8a47f81662b1 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:21.905760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 92/186] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:57:21.906907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d028a-47ba-4d93-981f-6903a82b2cb3 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:21.929477 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82079c85-9df4-4b1a-854a-2fc69e3cbea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:21.930688 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82079c85-9df4-4b1a-854a-2fc69e3cbea1 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:21.930998 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82079c85-9df4-4b1a-854a-2fc69e3cbea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:21.931271 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82079c85-9df4-4b1a-854a-2fc69e3cbea1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:21.941733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82079c85-9df4-4b1a-854a-2fc69e3cbea1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:21.949813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-82079c85-9df4-4b1a-854a-2fc69e3cbea1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:21.949813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:21.949813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:21.949813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:21.949813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:21.949813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:21.949813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:21.949813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:21.949813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:21.949813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:21.949813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:21.949813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:21.949813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:21.949813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:21.949813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:21.949813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:21.950736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:21.950736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:21.950736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:21.950736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:21.950736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:21.950736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:21.950736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:21.950736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:21.950736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:21.950736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:21.950736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:21.950736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:21.950736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:21.950736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:21.950736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:21.950736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:21.951567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:21.951567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:21.951567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:21.951567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:21.951567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:21.951567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:21.951567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:21.951567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:21.951567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:21.951567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:21.951567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:21.951567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:21.951567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 95/187] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:57:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:57:21.956351 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82079c85-9df4-4b1a-854a-2fc69e3cbea1 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:22.012138 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d028a-47ba-4d93-981f-6903a82b2cb3 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:22.045175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70797af5-bc15-4f2c-9cf6-aa7f46eeff91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:22.045971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70797af5-bc15-4f2c-9cf6-aa7f46eeff91 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:22.046286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70797af5-bc15-4f2c-9cf6-aa7f46eeff91 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:22.046551 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70797af5-bc15-4f2c-9cf6-aa7f46eeff91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:22.055407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82079c85-9df4-4b1a-854a-2fc69e3cbea1 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:22.057010 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70797af5-bc15-4f2c-9cf6-aa7f46eeff91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:22.064730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-70797af5-bc15-4f2c-9cf6-aa7f46eeff91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:22.064730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:22.064730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:22.064730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:22.064730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:22.064730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:22.064730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:22.064730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:22.064730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:22.064730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:22.064730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:22.064730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:22.064730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:22.064730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:22.064730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:22.064730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:22.065600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:22.065600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:22.065600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:22.065600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:22.065600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:22.065600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:22.065600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:22.065600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:22.065600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:22.065600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:22.065600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:22.065600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:22.065600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:22.065600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:22.065600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:22.065600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:22.066419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:22.066419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:22.066419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:22.066419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:22.066419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:22.066419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:22.066419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:22.066419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:22.066419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:22.066419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:22.066419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:22.066419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:22.066419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 93/188] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:57:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:22.067072 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70797af5-bc15-4f2c-9cf6-aa7f46eeff91 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:22.090283 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44859284-0850-4132-95a2-7516caf117f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:22.091079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44859284-0850-4132-95a2-7516caf117f6 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:22.091363 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44859284-0850-4132-95a2-7516caf117f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:22.091636 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44859284-0850-4132-95a2-7516caf117f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:22.092815 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44859284-0850-4132-95a2-7516caf117f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:22.094285 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44859284-0850-4132-95a2-7516caf117f6 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:22.096072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 96/189] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:57:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:22.096678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44859284-0850-4132-95a2-7516caf117f6 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:22.169982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70797af5-bc15-4f2c-9cf6-aa7f46eeff91 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:22.195083 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44859284-0850-4132-95a2-7516caf117f6 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:22.204263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895cf8c-f4db-4e93-b06d-a11e40d38e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:22.204935 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895cf8c-f4db-4e93-b06d-a11e40d38e73 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:22.205221 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895cf8c-f4db-4e93-b06d-a11e40d38e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:22.205499 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895cf8c-f4db-4e93-b06d-a11e40d38e73 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:22.206355 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895cf8c-f4db-4e93-b06d-a11e40d38e73 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:22.206666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895cf8c-f4db-4e93-b06d-a11e40d38e73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:22.208051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895cf8c-f4db-4e93-b06d-a11e40d38e73 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:22.216482 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 94/190] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:57:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:22.220988 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895cf8c-f4db-4e93-b06d-a11e40d38e73 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:22.229954 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734fe6e-1175-4aba-a746-aa764ae088cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:22.230876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734fe6e-1175-4aba-a746-aa764ae088cf None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:22.231217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734fe6e-1175-4aba-a746-aa764ae088cf None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:22.231809 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734fe6e-1175-4aba-a746-aa764ae088cf None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:22.232578 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734fe6e-1175-4aba-a746-aa764ae088cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:22.233208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734fe6e-1175-4aba-a746-aa764ae088cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:22.234869 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734fe6e-1175-4aba-a746-aa764ae088cf None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:22.250485 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 97/191] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:22.250485 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734fe6e-1175-4aba-a746-aa764ae088cf None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:22.318853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895cf8c-f4db-4e93-b06d-a11e40d38e73 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:22.351721 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734fe6e-1175-4aba-a746-aa764ae088cf None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:22.353041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7536c-eab0-4755-b4e9-8c6201c05d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:22.353804 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7536c-eab0-4755-b4e9-8c6201c05d2f None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:22.354219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7536c-eab0-4755-b4e9-8c6201c05d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:22.354559 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7536c-eab0-4755-b4e9-8c6201c05d2f None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:22.355403 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a7536c-eab0-4755-b4e9-8c6201c05d2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:22.356932 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a7536c-eab0-4755-b4e9-8c6201c05d2f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:22.383471 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085c806-4131-467f-9e66-7da9a08f87ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:22.384337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085c806-4131-467f-9e66-7da9a08f87ef None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:22.384617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085c806-4131-467f-9e66-7da9a08f87ef None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:22.384888 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085c806-4131-467f-9e66-7da9a08f87ef None admin] PATH_INFO: `/v3/users/a8217c0bdd0a4a7a8433de4d6410960f` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:22.399779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085c806-4131-467f-9e66-7da9a08f87ef None admin] RBAC: Authorizing `identity:get_user(user_id=a8217c0bdd0a4a7a8433de4d6410960f)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:22.401220 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085c806-4131-467f-9e66-7da9a08f87ef None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:22.402917 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 98/192] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:22 2022] GET /identity/v3/users/a8217c0bdd0a4a7a8433de4d6410960f => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:22.402917 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085c806-4131-467f-9e66-7da9a08f87ef None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:22.437282 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 95/193] 10.222.0.65 () {60 vars in 1132 bytes} [Tue May 10 19:57:22 2022] POST /identity/v3/users => generated 297 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:57:22.438301 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7536c-eab0-4755-b4e9-8c6201c05d2f None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:22.547495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085c806-4131-467f-9e66-7da9a08f87ef None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:22.558997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7536c-eab0-4755-b4e9-8c6201c05d2f None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:22.585707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7a512-f5ab-4cbc-b0ff-64ad2df555b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:22.587023 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7a512-f5ab-4cbc-b0ff-64ad2df555b6 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:22.587514 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7a512-f5ab-4cbc-b0ff-64ad2df555b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:22.587995 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7a512-f5ab-4cbc-b0ff-64ad2df555b6 None admin] PATH_INFO: `/v3/users/a8217c0bdd0a4a7a8433de4d6410960f` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:22.589508 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7a512-f5ab-4cbc-b0ff-64ad2df555b6 None admin] RBAC: Authorizing `identity:get_user(user_id=a8217c0bdd0a4a7a8433de4d6410960f)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:22.591306 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7a512-f5ab-4cbc-b0ff-64ad2df555b6 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:22.598107 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 99/194] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:22 2022] GET /identity/v3/users/a8217c0bdd0a4a7a8433de4d6410960f => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:22.599322 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7a512-f5ab-4cbc-b0ff-64ad2df555b6 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:22.599673 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ea11f-3127-404c-8190-74ead0f32ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:22.601050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ea11f-3127-404c-8190-74ead0f32ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:22.601489 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ea11f-3127-404c-8190-74ead0f32ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:22.601859 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ea11f-3127-404c-8190-74ead0f32ff1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:22.708643 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ea11f-3127-404c-8190-74ead0f32ff1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:22.712165 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ea11f-3127-404c-8190-74ead0f32ff1 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:22.731974 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7a512-f5ab-4cbc-b0ff-64ad2df555b6 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:22.786692 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240af9b6-477f-42bd-affd-4d2bba4f428d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:22.787540 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240af9b6-477f-42bd-affd-4d2bba4f428d None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:22.787920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240af9b6-477f-42bd-affd-4d2bba4f428d None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:22.788315 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240af9b6-477f-42bd-affd-4d2bba4f428d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:22.833818 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 96/195] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:22.841997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ea11f-3127-404c-8190-74ead0f32ff1 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:22.872982 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240af9b6-477f-42bd-affd-4d2bba4f428d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:22.874500 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240af9b6-477f-42bd-affd-4d2bba4f428d None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:22.944414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ea11f-3127-404c-8190-74ead0f32ff1 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:22.977126 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 100/196] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:22.982845 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75853c-decc-4e82-8b75-0f92f959b2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:22.983779 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75853c-decc-4e82-8b75-0f92f959b2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:22.984070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75853c-decc-4e82-8b75-0f92f959b2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:22.984347 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75853c-decc-4e82-8b75-0f92f959b2a6 None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:22.985615 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75853c-decc-4e82-8b75-0f92f959b2a6 None admin] RBAC: Authorizing `identity:get_project(project_id=775bb0dd255b49ca81b1452cbe667589)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:22.987093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75853c-decc-4e82-8b75-0f92f959b2a6 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:22.989078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 97/197] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:22 2022] GET /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:22.990033 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240af9b6-477f-42bd-affd-4d2bba4f428d None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:22.999333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75853c-decc-4e82-8b75-0f92f959b2a6 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:23.112199 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240af9b6-477f-42bd-affd-4d2bba4f428d None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:23.155047 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b77e6-6275-40c0-9e5f-4eb74a810613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:23.156309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b77e6-6275-40c0-9e5f-4eb74a810613 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:23.158140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b77e6-6275-40c0-9e5f-4eb74a810613 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:23.158645 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b77e6-6275-40c0-9e5f-4eb74a810613 None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:23.161031 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b77e6-6275-40c0-9e5f-4eb74a810613 None admin] RBAC: Authorizing `identity:get_project(project_id=775bb0dd255b49ca81b1452cbe667589)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:23.165833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b77e6-6275-40c0-9e5f-4eb74a810613 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:23.167851 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 101/198] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:22 2022] GET /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:23.173513 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b77e6-6275-40c0-9e5f-4eb74a810613 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:23.261474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75853c-decc-4e82-8b75-0f92f959b2a6 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:23.288141 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b77e6-6275-40c0-9e5f-4eb74a810613 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:23.326412 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa7ace-d92b-43ab-9f6b-e3f0070a6b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:23.330606 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa7ace-d92b-43ab-9f6b-e3f0070a6b36 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:23.330898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa7ace-d92b-43ab-9f6b-e3f0070a6b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:23.331170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa7ace-d92b-43ab-9f6b-e3f0070a6b36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:23.332027 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa7ace-d92b-43ab-9f6b-e3f0070a6b36 None admin] RBAC: Adding query filter params (role.id=5c87e804d0764309ba08ad6e06b23a87, scope.project.id=775bb0dd255b49ca81b1452cbe667589, user.id=a8217c0bdd0a4a7a8433de4d6410960f) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:23.332343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa7ace-d92b-43ab-9f6b-e3f0070a6b36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:23.333783 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa7ace-d92b-43ab-9f6b-e3f0070a6b36 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:23.336251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41890c-0b68-4683-a204-324359e26125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:23.336900 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41890c-0b68-4683-a204-324359e26125 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:23.337176 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41890c-0b68-4683-a204-324359e26125 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:23.337449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41890c-0b68-4683-a204-324359e26125 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:23.338325 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41890c-0b68-4683-a204-324359e26125 None admin] RBAC: Adding query filter params (role.id=4fbb0a346f5f4f36bb32664ee799db42, scope.project.id=775bb0dd255b49ca81b1452cbe667589, user.id=a8217c0bdd0a4a7a8433de4d6410960f) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:23.338637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41890c-0b68-4683-a204-324359e26125 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:23.340534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41890c-0b68-4683-a204-324359e26125 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:23.342924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 102/199] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:57:23 2022] GET /identity/v3/role_assignments?user.id=a8217c0bdd0a4a7a8433de4d6410960f&scope.project.id=775bb0dd255b49ca81b1452cbe667589&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:23.349593 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 98/200] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:57:22 2022] GET /identity/v3/role_assignments?user.id=a8217c0bdd0a4a7a8433de4d6410960f&scope.project.id=775bb0dd255b49ca81b1452cbe667589&role.id=4fbb0a346f5f4f36bb32664ee799db42 => generated 259 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:23.685478 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034d6c3-a68b-4268-ad44-e04728dcc533 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:23.686199 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034d6c3-a68b-4268-ad44-e04728dcc533 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:23.690579 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034d6c3-a68b-4268-ad44-e04728dcc533 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:23.692344 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 103/201] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:23.698885 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e79a6e-3a48-4034-bedb-d22ca0fba98d None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:23.699429 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e79a6e-3a48-4034-bedb-d22ca0fba98d None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:23.699901 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e79a6e-3a48-4034-bedb-d22ca0fba98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:23.723108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-53e79a6e-3a48-4034-bedb-d22ca0fba98d None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:23.724643 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-53e79a6e-3a48-4034-bedb-d22ca0fba98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:23.800210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 99/202] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:23.835615 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034d6c3-a68b-4268-ad44-e04728dcc533 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:23.969134 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034d6c3-a68b-4268-ad44-e04728dcc533 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:24.014834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0266a2e-3b93-4b31-8cf6-7e4f4812eecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:24.015642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0266a2e-3b93-4b31-8cf6-7e4f4812eecc None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:24.015929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0266a2e-3b93-4b31-8cf6-7e4f4812eecc None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:24.016202 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0266a2e-3b93-4b31-8cf6-7e4f4812eecc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:24.027339 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0266a2e-3b93-4b31-8cf6-7e4f4812eecc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:24.035165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-e0266a2e-3b93-4b31-8cf6-7e4f4812eecc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 19:57:24.035165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:24.035165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:24.035165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:24.035165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:24.035165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:24.035165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:24.035165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:24.035165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:24.035165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:24.035165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:24.035165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:24.035165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:24.035165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:24.035165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:24.035165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:24.036118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:24.036118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:24.036118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:24.036118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:24.036118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:24.036118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:24.036118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:24.036118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:24.036118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:24.036118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:24.036118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:24.036118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:24.036118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:24.036118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:24.036118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:24.036118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:24.036920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:24.036920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:24.036920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:24.036920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:24.036920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:24.036920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:24.036920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:24.036920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:24.036920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:24.036920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:24.036920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 19:57:24.036920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:24.036920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 104/203] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:24.043385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e79a6e-3a48-4034-bedb-d22ca0fba98d None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:24.213705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e79a6e-3a48-4034-bedb-d22ca0fba98d None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:24.255925 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979243ff-1c02-4a13-a5d3-6827ba30f229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:24.257057 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979243ff-1c02-4a13-a5d3-6827ba30f229 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:24.257337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979243ff-1c02-4a13-a5d3-6827ba30f229 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:24.257609 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979243ff-1c02-4a13-a5d3-6827ba30f229 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:24.258469 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979243ff-1c02-4a13-a5d3-6827ba30f229 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:24.258786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979243ff-1c02-4a13-a5d3-6827ba30f229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:24.260191 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979243ff-1c02-4a13-a5d3-6827ba30f229 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:24.274767 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 100/204] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:24.280091 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0266a2e-3b93-4b31-8cf6-7e4f4812eecc None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:24.488315 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0266a2e-3b93-4b31-8cf6-7e4f4812eecc None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:24.533507 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54669e99-3d86-4ef2-a512-b9a1a4eb4444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:24.534335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54669e99-3d86-4ef2-a512-b9a1a4eb4444 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:24.534625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54669e99-3d86-4ef2-a512-b9a1a4eb4444 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:24.534905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54669e99-3d86-4ef2-a512-b9a1a4eb4444 None admin] PATH_INFO: `/v3/users/bb370646880f4ad2b7b216d60a97e638` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:24.536118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54669e99-3d86-4ef2-a512-b9a1a4eb4444 None admin] RBAC: Authorizing `identity:get_user(user_id=bb370646880f4ad2b7b216d60a97e638)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:24.537585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54669e99-3d86-4ef2-a512-b9a1a4eb4444 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:24.539248 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 105/205] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:24 2022] GET /identity/v3/users/bb370646880f4ad2b7b216d60a97e638 => generated 289 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:24.547139 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979243ff-1c02-4a13-a5d3-6827ba30f229 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:24.692299 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979243ff-1c02-4a13-a5d3-6827ba30f229 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:24.733629 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43fdeb-77cc-4f77-a75a-e32d49f22fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:24.734358 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43fdeb-77cc-4f77-a75a-e32d49f22fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:24.734647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43fdeb-77cc-4f77-a75a-e32d49f22fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:24.734921 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43fdeb-77cc-4f77-a75a-e32d49f22fc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:24.805783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43fdeb-77cc-4f77-a75a-e32d49f22fc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:24.807251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43fdeb-77cc-4f77-a75a-e32d49f22fc8 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:24.909540 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 101/206] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:24.915301 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54669e99-3d86-4ef2-a512-b9a1a4eb4444 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:25.069685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54669e99-3d86-4ef2-a512-b9a1a4eb4444 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:25.121145 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84ae19-9aaa-4678-9bbd-ef3b48b17755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:25.122003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84ae19-9aaa-4678-9bbd-ef3b48b17755 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:25.122311 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84ae19-9aaa-4678-9bbd-ef3b48b17755 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:25.122589 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84ae19-9aaa-4678-9bbd-ef3b48b17755 None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:25.123801 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84ae19-9aaa-4678-9bbd-ef3b48b17755 None admin] RBAC: Authorizing `identity:get_project(project_id=675b1b490743424e9d44dd2e484dfa95)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:25.125247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84ae19-9aaa-4678-9bbd-ef3b48b17755 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:25.127297 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 106/207] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:24 2022] GET /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:25.133545 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43fdeb-77cc-4f77-a75a-e32d49f22fc8 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:25.300405 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43fdeb-77cc-4f77-a75a-e32d49f22fc8 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:25.342741 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf441c3-b569-4f9f-8547-2ebd917fe3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:25.343735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf441c3-b569-4f9f-8547-2ebd917fe3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:25.344237 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf441c3-b569-4f9f-8547-2ebd917fe3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:25.344731 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf441c3-b569-4f9f-8547-2ebd917fe3c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:25.345769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf441c3-b569-4f9f-8547-2ebd917fe3c8 None admin] RBAC: Adding query filter params (role.id=f7a4054724ca4163960cf635d13b055f, scope.project.id=675b1b490743424e9d44dd2e484dfa95, user.id=bb370646880f4ad2b7b216d60a97e638) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:25.346309 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf441c3-b569-4f9f-8547-2ebd917fe3c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:25.347950 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf441c3-b569-4f9f-8547-2ebd917fe3c8 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:25.358601 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 102/208] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:57:25 2022] GET /identity/v3/role_assignments?user.id=bb370646880f4ad2b7b216d60a97e638&scope.project.id=675b1b490743424e9d44dd2e484dfa95&role.id=f7a4054724ca4163960cf635d13b055f => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:25.531448 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ce599-c078-4118-aa8b-ed20b2c64499 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:25.532428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ce599-c078-4118-aa8b-ed20b2c64499 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:25.532909 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ce599-c078-4118-aa8b-ed20b2c64499 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:25.534610 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 107/209] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:25.540618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c6b3a-fd55-49b0-8589-8bb4a65cf439 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:25.540913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c6b3a-fd55-49b0-8589-8bb4a65cf439 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:25.541198 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c6b3a-fd55-49b0-8589-8bb4a65cf439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:25.566596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-b07c6b3a-fd55-49b0-8589-8bb4a65cf439 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:25.567826 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b07c6b3a-fd55-49b0-8589-8bb4a65cf439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:25.589533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b93f5-80fe-4bc8-a541-6f0ef2c87d84 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:25.590194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b93f5-80fe-4bc8-a541-6f0ef2c87d84 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:25.590676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b93f5-80fe-4bc8-a541-6f0ef2c87d84 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:25.592244 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 108/210] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:57:25.600980 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2063992-5b72-43ed-b1fa-70e5f410015b None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:25.601185 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2063992-5b72-43ed-b1fa-70e5f410015b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:25.601628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2063992-5b72-43ed-b1fa-70e5f410015b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:25.625067 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-d2063992-5b72-43ed-b1fa-70e5f410015b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:25.626568 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-d2063992-5b72-43ed-b1fa-70e5f410015b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:25.682065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 103/211] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:57:25.706896 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 109/212] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:57:25.711565 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c6b3a-fd55-49b0-8589-8bb4a65cf439 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:25.732801 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2063992-5b72-43ed-b1fa-70e5f410015b None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:25.826439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c6b3a-fd55-49b0-8589-8bb4a65cf439 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:25.839241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2063992-5b72-43ed-b1fa-70e5f410015b None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:25.862908 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae6d7e-c4a1-4e63-9f32-ccfe44acff84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:25.863994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae6d7e-c4a1-4e63-9f32-ccfe44acff84 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:25.864412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae6d7e-c4a1-4e63-9f32-ccfe44acff84 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:25.864876 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae6d7e-c4a1-4e63-9f32-ccfe44acff84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:25.866962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae6d7e-c4a1-4e63-9f32-ccfe44acff84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:25.868842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae6d7e-c4a1-4e63-9f32-ccfe44acff84 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:25.870955 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 104/213] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:57:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:25.875962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae6d7e-c4a1-4e63-9f32-ccfe44acff84 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:25.883453 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec2cec-17c3-4cdf-8d88-2d83367f96b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:25.884543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec2cec-17c3-4cdf-8d88-2d83367f96b9 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:25.884974 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec2cec-17c3-4cdf-8d88-2d83367f96b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:25.885397 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec2cec-17c3-4cdf-8d88-2d83367f96b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:25.896391 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec2cec-17c3-4cdf-8d88-2d83367f96b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:25.905032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-73ec2cec-17c3-4cdf-8d88-2d83367f96b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:25.905032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:25.905032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:25.905032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:25.905032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:25.905032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:25.905032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:25.905032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:25.905032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:25.905032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:25.905032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:25.905032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:25.905032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:25.905032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:25.905032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:25.905032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:25.906191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:25.906191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:25.906191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:25.906191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:25.906191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:25.906191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:25.906191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:25.906191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:25.906191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:25.906191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:25.906191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:25.906191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:25.906191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:25.906191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:25.906191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:25.906191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:25.907238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:25.907238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:25.907238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:25.907238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:25.907238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:25.907238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:25.907238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:25.907238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:25.907238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:25.907238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:25.907238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:25.907238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:25.909377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 110/214] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:57:25.923295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec2cec-17c3-4cdf-8d88-2d83367f96b9 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:25.998472 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae6d7e-c4a1-4e63-9f32-ccfe44acff84 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:26.033686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc29e19-cfb8-46aa-b637-6781ade04d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:26.034379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec2cec-17c3-4cdf-8d88-2d83367f96b9 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:26.034490 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc29e19-cfb8-46aa-b637-6781ade04d60 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:26.034658 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc29e19-cfb8-46aa-b637-6781ade04d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:26.034939 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc29e19-cfb8-46aa-b637-6781ade04d60 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:26.035655 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc29e19-cfb8-46aa-b637-6781ade04d60 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:26.037153 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc29e19-cfb8-46aa-b637-6781ade04d60 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:26.067349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50826201-8913-4b42-a6a1-163248dece82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:26.068026 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50826201-8913-4b42-a6a1-163248dece82 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:26.068304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50826201-8913-4b42-a6a1-163248dece82 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:26.068604 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50826201-8913-4b42-a6a1-163248dece82 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:26.071550 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50826201-8913-4b42-a6a1-163248dece82 None admin] RBAC: Adding query filter params (name=member) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:26.071550 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50826201-8913-4b42-a6a1-163248dece82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:26.071550 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50826201-8913-4b42-a6a1-163248dece82 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:26.078912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 111/215] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:26.080252 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffede8-7ac5-427f-80d7-56ff88c898f4 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:26.080541 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffede8-7ac5-427f-80d7-56ff88c898f4 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:26.080812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffede8-7ac5-427f-80d7-56ff88c898f4 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:26.082214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 112/216] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:26.086999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffede8-7ac5-427f-80d7-56ff88c898f4 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:26.149191 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 105/217] 10.222.0.65 () {60 vars in 1132 bytes} [Tue May 10 19:57:25 2022] POST /identity/v3/users => generated 297 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:57:26.150869 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5705fb-57d6-42d4-8f70-75d45b779ed6 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:26.151160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5705fb-57d6-42d4-8f70-75d45b779ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:26.151436 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5705fb-57d6-42d4-8f70-75d45b779ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:26.169658 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-4e5705fb-57d6-42d4-8f70-75d45b779ed6 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:26.170916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-4e5705fb-57d6-42d4-8f70-75d45b779ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:26.202265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffede8-7ac5-427f-80d7-56ff88c898f4 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:26.238769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb174c2d-1a0b-44b9-b8be-12f7b4ad10a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:26.239750 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb174c2d-1a0b-44b9-b8be-12f7b4ad10a0 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:26.240039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb174c2d-1a0b-44b9-b8be-12f7b4ad10a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:26.240409 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb174c2d-1a0b-44b9-b8be-12f7b4ad10a0 None admin] PATH_INFO: `/v3/users/f37c0defef304bc69cc6edbfc44d6399` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:26.249630 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 106/218] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:26.249630 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53194b4-496d-4712-b262-a450e1ab70ee None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:26.249630 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53194b4-496d-4712-b262-a450e1ab70ee None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:26.249630 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53194b4-496d-4712-b262-a450e1ab70ee None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:26.249630 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 107/219] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:26.251295 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d6c0b-4588-4e22-be49-b0979d60b0ef None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:26.251500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d6c0b-4588-4e22-be49-b0979d60b0ef None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:26.251786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d6c0b-4588-4e22-be49-b0979d60b0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:26.267767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb174c2d-1a0b-44b9-b8be-12f7b4ad10a0 None admin] RBAC: Authorizing `identity:get_user(user_id=f37c0defef304bc69cc6edbfc44d6399)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:26.269620 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb174c2d-1a0b-44b9-b8be-12f7b4ad10a0 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:26.272156 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 113/220] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:26 2022] GET /identity/v3/users/f37c0defef304bc69cc6edbfc44d6399 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:26.272861 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb174c2d-1a0b-44b9-b8be-12f7b4ad10a0 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:26.275439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-a26d6c0b-4588-4e22-be49-b0979d60b0ef None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:26.276517 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a26d6c0b-4588-4e22-be49-b0979d60b0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:26.345813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 108/221] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:57:26.347375 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d6c0b-4588-4e22-be49-b0979d60b0ef None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:26.398497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb174c2d-1a0b-44b9-b8be-12f7b4ad10a0 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:26.445699 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73158d78-5802-4861-be3b-72bbd5d10f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:26.446629 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73158d78-5802-4861-be3b-72bbd5d10f90 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:26.446920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73158d78-5802-4861-be3b-72bbd5d10f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:26.447199 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73158d78-5802-4861-be3b-72bbd5d10f90 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:26.469376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73158d78-5802-4861-be3b-72bbd5d10f90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:26.478553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-73158d78-5802-4861-be3b-72bbd5d10f90 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:26.478553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:26.478553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:26.478553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:26.478553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:26.478553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:26.478553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:26.478553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:26.478553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:26.478553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:26.478553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:26.478553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:26.478553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:26.478553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:26.478553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:26.478553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:26.480222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:26.480222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:26.480222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:26.480222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:26.480222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:26.480222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:26.480222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:26.480222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:26.480222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:26.480222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:26.480222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:26.480222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:26.480222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:26.480222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:26.480222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:26.480222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:26.481501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:26.481501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:26.481501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:26.481501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:26.481501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:26.481501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:26.481501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:26.481501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:26.481501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:26.481501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:26.481501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:26.481501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:26.482188 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d6c0b-4588-4e22-be49-b0979d60b0ef None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:26.482285 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 114/222] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:57:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:26.482285 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73158d78-5802-4861-be3b-72bbd5d10f90 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:26.527099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623cdc8b-4c38-4d88-94c0-3b83a9b5bcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:26.527756 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cdc8b-4c38-4d88-94c0-3b83a9b5bcf2 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:26.528076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cdc8b-4c38-4d88-94c0-3b83a9b5bcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:26.528320 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cdc8b-4c38-4d88-94c0-3b83a9b5bcf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:26.597920 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623cdc8b-4c38-4d88-94c0-3b83a9b5bcf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:26.599767 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623cdc8b-4c38-4d88-94c0-3b83a9b5bcf2 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:26.618413 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73158d78-5802-4861-be3b-72bbd5d10f90 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:26.653586 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c04b24-f99b-4cde-8461-b88538024e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:26.654514 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c04b24-f99b-4cde-8461-b88538024e86 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:26.654800 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c04b24-f99b-4cde-8461-b88538024e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:26.655079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c04b24-f99b-4cde-8461-b88538024e86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:26.666015 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c04b24-f99b-4cde-8461-b88538024e86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:26.673876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-b2c04b24-f99b-4cde-8461-b88538024e86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:26.673876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:26.673876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:26.673876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:26.673876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:26.673876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:26.673876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:26.673876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:26.673876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:26.673876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:26.673876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:26.673876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:26.673876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:26.673876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:26.673876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:26.673876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:26.674820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:26.674820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:26.674820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:26.674820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:26.674820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:26.674820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:26.674820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:26.674820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:26.674820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:26.674820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:26.674820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:26.674820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:26.674820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:26.674820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:26.674820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:26.674820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:26.675817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:26.675817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:26.675817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:26.675817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:26.675817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:26.675817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:26.675817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:26.675817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:26.675817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:26.675817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:26.675817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:26.675817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:26.675817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 115/223] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:57:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:26.676453 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c04b24-f99b-4cde-8461-b88538024e86 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:26.720972 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 109/224] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:26.721640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623cdc8b-4c38-4d88-94c0-3b83a9b5bcf2 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:26.789752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c04b24-f99b-4cde-8461-b88538024e86 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:26.824365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbff120-956e-43a4-9b4d-71f26f81d043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:26.825036 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbff120-956e-43a4-9b4d-71f26f81d043 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:26.825318 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbff120-956e-43a4-9b4d-71f26f81d043 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:26.825593 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbff120-956e-43a4-9b4d-71f26f81d043 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:26.826381 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbff120-956e-43a4-9b4d-71f26f81d043 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:26.826688 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbff120-956e-43a4-9b4d-71f26f81d043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:26.827132 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623cdc8b-4c38-4d88-94c0-3b83a9b5bcf2 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:26.828209 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbff120-956e-43a4-9b4d-71f26f81d043 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:26.841225 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 116/225] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:57:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:26.842447 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbff120-956e-43a4-9b4d-71f26f81d043 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:26.870413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e77341-6c35-43dc-833f-ca99a636ccb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:26.872014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e77341-6c35-43dc-833f-ca99a636ccb2 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:26.872784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e77341-6c35-43dc-833f-ca99a636ccb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:26.873286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e77341-6c35-43dc-833f-ca99a636ccb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:26.875420 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e77341-6c35-43dc-833f-ca99a636ccb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:26.876181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e77341-6c35-43dc-833f-ca99a636ccb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:26.878937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e77341-6c35-43dc-833f-ca99a636ccb2 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:26.890544 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 110/226] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:26.891751 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e77341-6c35-43dc-833f-ca99a636ccb2 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:26.961245 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbff120-956e-43a4-9b4d-71f26f81d043 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:26.996075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6bfac-f54d-4329-896a-7c2ce0c1f299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:26.996944 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6bfac-f54d-4329-896a-7c2ce0c1f299 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:26.997295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6bfac-f54d-4329-896a-7c2ce0c1f299 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:26.997639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6bfac-f54d-4329-896a-7c2ce0c1f299 None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:26.998995 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6bfac-f54d-4329-896a-7c2ce0c1f299 None admin] RBAC: Authorizing `identity:get_project(project_id=775bb0dd255b49ca81b1452cbe667589)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:27.000527 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6bfac-f54d-4329-896a-7c2ce0c1f299 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:27.002319 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 117/227] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:26 2022] GET /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:27.003143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6bfac-f54d-4329-896a-7c2ce0c1f299 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:27.009880 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e77341-6c35-43dc-833f-ca99a636ccb2 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:27.047007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971f5f4-be51-4a84-ad44-8bb3573b51cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:27.047899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971f5f4-be51-4a84-ad44-8bb3573b51cf None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:27.048187 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971f5f4-be51-4a84-ad44-8bb3573b51cf None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:27.048465 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971f5f4-be51-4a84-ad44-8bb3573b51cf None admin] PATH_INFO: `/v3/users/a8217c0bdd0a4a7a8433de4d6410960f` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:27.049697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971f5f4-be51-4a84-ad44-8bb3573b51cf None admin] RBAC: Authorizing `identity:get_user(user_id=a8217c0bdd0a4a7a8433de4d6410960f)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:27.051246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971f5f4-be51-4a84-ad44-8bb3573b51cf None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:27.053700 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 111/228] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:26 2022] GET /identity/v3/users/a8217c0bdd0a4a7a8433de4d6410960f => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:27.054342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971f5f4-be51-4a84-ad44-8bb3573b51cf None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:27.111514 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6bfac-f54d-4329-896a-7c2ce0c1f299 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:27.147312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4b34e-db66-4784-b55f-aaf1de8a990a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:27.148184 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4b34e-db66-4784-b55f-aaf1de8a990a None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:27.148466 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4b34e-db66-4784-b55f-aaf1de8a990a None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:27.148739 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4b34e-db66-4784-b55f-aaf1de8a990a None admin] PATH_INFO: `/v3/users/a8217c0bdd0a4a7a8433de4d6410960f` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:27.149958 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4b34e-db66-4784-b55f-aaf1de8a990a None admin] RBAC: Authorizing `identity:get_user(user_id=a8217c0bdd0a4a7a8433de4d6410960f)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:27.151426 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4b34e-db66-4784-b55f-aaf1de8a990a None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:27.157963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 118/229] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:27 2022] GET /identity/v3/users/a8217c0bdd0a4a7a8433de4d6410960f => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:27.158899 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4b34e-db66-4784-b55f-aaf1de8a990a None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:27.170580 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971f5f4-be51-4a84-ad44-8bb3573b51cf None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:27.208272 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80dd82-2abc-4386-bb6f-b25a5dcb885c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:27.209020 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80dd82-2abc-4386-bb6f-b25a5dcb885c None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:27.209312 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80dd82-2abc-4386-bb6f-b25a5dcb885c None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:27.209590 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80dd82-2abc-4386-bb6f-b25a5dcb885c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:27.210472 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80dd82-2abc-4386-bb6f-b25a5dcb885c None admin] RBAC: Adding query filter params (role.id=5f62ce25636443f8a5add67f208afe62, scope.project.id=775bb0dd255b49ca81b1452cbe667589, user.id=f37c0defef304bc69cc6edbfc44d6399) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:27.210785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80dd82-2abc-4386-bb6f-b25a5dcb885c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:27.212240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80dd82-2abc-4386-bb6f-b25a5dcb885c None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:27.218165 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 112/230] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:57:27 2022] GET /identity/v3/role_assignments?user.id=f37c0defef304bc69cc6edbfc44d6399&scope.project.id=775bb0dd255b49ca81b1452cbe667589&role.id=5f62ce25636443f8a5add67f208afe62 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:27.219034 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80dd82-2abc-4386-bb6f-b25a5dcb885c None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:27.266558 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4b34e-db66-4784-b55f-aaf1de8a990a None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:27.303614 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2a748-6ea7-4366-95c9-7aa07a01735a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:27.304472 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2a748-6ea7-4366-95c9-7aa07a01735a None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:27.304753 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2a748-6ea7-4366-95c9-7aa07a01735a None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:27.305027 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2a748-6ea7-4366-95c9-7aa07a01735a None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:27.306267 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2a748-6ea7-4366-95c9-7aa07a01735a None admin] RBAC: Authorizing `identity:get_project(project_id=775bb0dd255b49ca81b1452cbe667589)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:27.307713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2a748-6ea7-4366-95c9-7aa07a01735a None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:27.309554 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 119/231] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:27 2022] GET /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:27.320096 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2a748-6ea7-4366-95c9-7aa07a01735a None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:27.339821 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80dd82-2abc-4386-bb6f-b25a5dcb885c None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:27.375701 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae4f57-9671-4111-a12e-7afbad1c8e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:27.376570 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae4f57-9671-4111-a12e-7afbad1c8e33 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:27.376856 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae4f57-9671-4111-a12e-7afbad1c8e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:27.377130 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae4f57-9671-4111-a12e-7afbad1c8e33 None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:27.378464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae4f57-9671-4111-a12e-7afbad1c8e33 None admin] RBAC: Authorizing `identity:get_project(project_id=775bb0dd255b49ca81b1452cbe667589)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:27.379973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae4f57-9671-4111-a12e-7afbad1c8e33 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:27.382372 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 113/232] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:27 2022] GET /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:27.388093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae4f57-9671-4111-a12e-7afbad1c8e33 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:27.501733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae4f57-9671-4111-a12e-7afbad1c8e33 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:27.539617 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe83a7-79ae-40d9-9290-625aab66e2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:27.540438 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe83a7-79ae-40d9-9290-625aab66e2a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:27.540724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe83a7-79ae-40d9-9290-625aab66e2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:27.541004 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe83a7-79ae-40d9-9290-625aab66e2a2 None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589/users/a8217c0bdd0a4a7a8433de4d6410960f/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:27.543087 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe83a7-79ae-40d9-9290-625aab66e2a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=775bb0dd255b49ca81b1452cbe667589, user_id=a8217c0bdd0a4a7a8433de4d6410960f, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:27.544537 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe83a7-79ae-40d9-9290-625aab66e2a2 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:27.546563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2a748-6ea7-4366-95c9-7aa07a01735a None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:27.589401 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d389ba1-8e65-4fbc-a28a-dea721aafaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:27.590374 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d389ba1-8e65-4fbc-a28a-dea721aafaa4 None admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:27.590663 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d389ba1-8e65-4fbc-a28a-dea721aafaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:27.590940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d389ba1-8e65-4fbc-a28a-dea721aafaa4 None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589/users/a8217c0bdd0a4a7a8433de4d6410960f/roles/4fbb0a346f5f4f36bb32664ee799db42` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:27.593449 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d389ba1-8e65-4fbc-a28a-dea721aafaa4 None admin] RBAC: Authorizing `identity:create_grant(project_id=775bb0dd255b49ca81b1452cbe667589, user_id=a8217c0bdd0a4a7a8433de4d6410960f, role_id=4fbb0a346f5f4f36bb32664ee799db42)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:27.594937 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d389ba1-8e65-4fbc-a28a-dea721aafaa4 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:27.622513 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 114/233] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:57:27 2022] PUT /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589/users/a8217c0bdd0a4a7a8433de4d6410960f/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 19:57:27.682929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 120/234] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:57:27 2022] PUT /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589/users/a8217c0bdd0a4a7a8433de4d6410960f/roles/4fbb0a346f5f4f36bb32664ee799db42 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 19:57:28.758956 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5dc2cf-2988-448e-b69a-2a90bcecf8a8 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:28.759971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5dc2cf-2988-448e-b69a-2a90bcecf8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:28.760477 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5dc2cf-2988-448e-b69a-2a90bcecf8a8 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:28.762056 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 115/235] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:28.769341 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b73140-96bb-469c-a1cf-d4428e72f11e None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:28.769813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b73140-96bb-469c-a1cf-d4428e72f11e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:28.769891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b73140-96bb-469c-a1cf-d4428e72f11e None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:28.771958 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 121/236] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:28.775871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9acdc-5a7f-4317-ac0c-1f1e64cc19ae None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:28.776160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9acdc-5a7f-4317-ac0c-1f1e64cc19ae None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:28.776440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9acdc-5a7f-4317-ac0c-1f1e64cc19ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:28.785427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1eff5-1afa-4128-ac4a-434f8e9de061 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:28.785971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1eff5-1afa-4128-ac4a-434f8e9de061 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:28.786489 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1eff5-1afa-4128-ac4a-434f8e9de061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:28.795020 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-12d9acdc-5a7f-4317-ac0c-1f1e64cc19ae None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:28.796266 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-12d9acdc-5a7f-4317-ac0c-1f1e64cc19ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:28.821802 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-b9d1eff5-1afa-4128-ac4a-434f8e9de061 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:28.823056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b9d1eff5-1afa-4128-ac4a-434f8e9de061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:28.937642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 122/237] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:28.948985 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 116/238] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:28.967324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1eff5-1afa-4128-ac4a-434f8e9de061 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:28.987524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9acdc-5a7f-4317-ac0c-1f1e64cc19ae None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:29.072276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1eff5-1afa-4128-ac4a-434f8e9de061 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:29.113271 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e500b-45e6-4a2e-9b6c-b479d93c853f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:29.117043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e500b-45e6-4a2e-9b6c-b479d93c853f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:29.117333 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e500b-45e6-4a2e-9b6c-b479d93c853f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:29.117868 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e500b-45e6-4a2e-9b6c-b479d93c853f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:29.119086 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e500b-45e6-4a2e-9b6c-b479d93c853f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:29.120523 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e500b-45e6-4a2e-9b6c-b479d93c853f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:29.125981 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 123/239] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:57:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:29.132038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e500b-45e6-4a2e-9b6c-b479d93c853f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:29.188614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9acdc-5a7f-4317-ac0c-1f1e64cc19ae None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:29.241988 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3911f-defe-4b01-a740-93dbb84dd3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:29.243073 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3911f-defe-4b01-a740-93dbb84dd3af None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:29.243358 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3911f-defe-4b01-a740-93dbb84dd3af None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:29.243631 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3911f-defe-4b01-a740-93dbb84dd3af None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:29.251806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e500b-45e6-4a2e-9b6c-b479d93c853f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:29.260568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3911f-defe-4b01-a740-93dbb84dd3af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:29.276687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-19c3911f-defe-4b01-a740-93dbb84dd3af None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 19:57:29.276687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:29.276687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:29.276687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:29.276687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:29.276687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:29.276687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:29.276687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:29.276687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:29.276687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:29.276687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:29.276687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:29.276687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:29.276687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:29.276687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:29.276687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:29.277641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:29.277641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:29.277641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:29.277641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:29.277641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:29.277641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:29.277641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:29.277641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:29.277641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:29.277641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:29.277641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:29.277641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:29.277641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:29.277641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:29.277641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:29.277641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:29.278930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:29.278930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:29.278930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:29.278930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:29.278930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:29.278930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:29.278930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:29.278930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:29.278930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:29.278930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:29.278930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 19:57:29.278930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:29.278930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 117/240] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:57:29.284160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3911f-defe-4b01-a740-93dbb84dd3af None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:29.293957 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab0730-b71c-49a4-9554-9c3799c3ad7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:29.294716 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab0730-b71c-49a4-9554-9c3799c3ad7d None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:29.295006 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab0730-b71c-49a4-9554-9c3799c3ad7d None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:29.303703 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab0730-b71c-49a4-9554-9c3799c3ad7d None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:29.303703 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab0730-b71c-49a4-9554-9c3799c3ad7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:29.303703 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab0730-b71c-49a4-9554-9c3799c3ad7d None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:29.385307 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3911f-defe-4b01-a740-93dbb84dd3af None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:29.408948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 124/241] 10.222.0.65 () {60 vars in 1132 bytes} [Tue May 10 19:57:29 2022] POST /identity/v3/users => generated 293 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:57:29.434966 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e3b46-89fe-4a51-8681-c93dd2b6506b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:29.435664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e3b46-89fe-4a51-8681-c93dd2b6506b None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:29.435949 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e3b46-89fe-4a51-8681-c93dd2b6506b None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:29.436223 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e3b46-89fe-4a51-8681-c93dd2b6506b None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:29.436997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e3b46-89fe-4a51-8681-c93dd2b6506b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:29.437304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e3b46-89fe-4a51-8681-c93dd2b6506b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:29.438733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e3b46-89fe-4a51-8681-c93dd2b6506b None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:29.448012 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 118/242] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:29.453607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab0730-b71c-49a4-9554-9c3799c3ad7d None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:29.561297 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab0730-b71c-49a4-9554-9c3799c3ad7d None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:29.602113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d00a39-7a49-4905-aa88-8d2d688b1d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:29.602987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d00a39-7a49-4905-aa88-8d2d688b1d04 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:29.603267 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d00a39-7a49-4905-aa88-8d2d688b1d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:29.603538 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d00a39-7a49-4905-aa88-8d2d688b1d04 None admin] PATH_INFO: `/v3/users/6ccfbf763b834334b259636e317cb678` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:29.631012 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d00a39-7a49-4905-aa88-8d2d688b1d04 None admin] RBAC: Authorizing `identity:get_user(user_id=6ccfbf763b834334b259636e317cb678)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:29.632517 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d00a39-7a49-4905-aa88-8d2d688b1d04 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:29.634321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 125/243] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:29 2022] GET /identity/v3/users/6ccfbf763b834334b259636e317cb678 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:29.639953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e3b46-89fe-4a51-8681-c93dd2b6506b None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:29.761287 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e3b46-89fe-4a51-8681-c93dd2b6506b None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:29.799749 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbf351-6ca7-4c87-966b-aaaa6088b8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:29.800777 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbf351-6ca7-4c87-966b-aaaa6088b8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:29.801267 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbf351-6ca7-4c87-966b-aaaa6088b8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:29.801745 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbf351-6ca7-4c87-966b-aaaa6088b8d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:29.887522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dbf351-6ca7-4c87-966b-aaaa6088b8d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:29.889348 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dbf351-6ca7-4c87-966b-aaaa6088b8d6 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:30.008866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 119/244] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:30.014048 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d00a39-7a49-4905-aa88-8d2d688b1d04 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:30.140080 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d00a39-7a49-4905-aa88-8d2d688b1d04 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:30.179960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daea30e-8c0f-4d3e-b478-7dc26c52b848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:30.180833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daea30e-8c0f-4d3e-b478-7dc26c52b848 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:30.181112 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daea30e-8c0f-4d3e-b478-7dc26c52b848 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:30.181382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daea30e-8c0f-4d3e-b478-7dc26c52b848 None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:30.182784 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daea30e-8c0f-4d3e-b478-7dc26c52b848 None admin] RBAC: Authorizing `identity:get_project(project_id=775bb0dd255b49ca81b1452cbe667589)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:30.184243 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daea30e-8c0f-4d3e-b478-7dc26c52b848 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:30.186275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 126/245] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:30 2022] GET /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:30.192454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbf351-6ca7-4c87-966b-aaaa6088b8d6 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:30.279451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d20c9d-8f16-4451-a0ca-eebbc74e1680 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:30.279740 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d20c9d-8f16-4451-a0ca-eebbc74e1680 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:30.279933 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d20c9d-8f16-4451-a0ca-eebbc74e1680 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:30.281436 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 127/246] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:57:30.287020 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163c28e-3274-41c5-a14b-c0965a5bdfcd None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:30.287364 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163c28e-3274-41c5-a14b-c0965a5bdfcd None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:30.287584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163c28e-3274-41c5-a14b-c0965a5bdfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:30.311428 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbf351-6ca7-4c87-966b-aaaa6088b8d6 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:30.339638 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-0163c28e-3274-41c5-a14b-c0965a5bdfcd None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:30.340837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-0163c28e-3274-41c5-a14b-c0965a5bdfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:30.363787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f41cd6-f9dd-4110-af66-83ce06f1c1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:30.364754 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f41cd6-f9dd-4110-af66-83ce06f1c1fc None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:30.365078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f41cd6-f9dd-4110-af66-83ce06f1c1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:30.365359 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f41cd6-f9dd-4110-af66-83ce06f1c1fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:30.366404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f41cd6-f9dd-4110-af66-83ce06f1c1fc None admin] RBAC: Adding query filter params (role.id=f7a4054724ca4163960cf635d13b055f, scope.project.id=775bb0dd255b49ca81b1452cbe667589, user.id=6ccfbf763b834334b259636e317cb678) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:30.366719 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f41cd6-f9dd-4110-af66-83ce06f1c1fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:30.368421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f41cd6-f9dd-4110-af66-83ce06f1c1fc None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:30.374837 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 120/247] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:57:30 2022] GET /identity/v3/role_assignments?user.id=6ccfbf763b834334b259636e317cb678&scope.project.id=775bb0dd255b49ca81b1452cbe667589&role.id=f7a4054724ca4163960cf635d13b055f => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:30.376303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c297d-cba5-4ce9-870d-efaf099f1864 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:30.376649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c297d-cba5-4ce9-870d-efaf099f1864 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:30.376932 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c297d-cba5-4ce9-870d-efaf099f1864 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:30.383461 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 121/248] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:30.392699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34138786-5b36-495a-9e8e-55edce2e6246 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:30.393008 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34138786-5b36-495a-9e8e-55edce2e6246 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:30.393258 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34138786-5b36-495a-9e8e-55edce2e6246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:30.428983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-34138786-5b36-495a-9e8e-55edce2e6246 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:30.428983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-34138786-5b36-495a-9e8e-55edce2e6246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:30.486233 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 128/249] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:30.493445 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d874a-a19a-4423-94d9-8ce46b893543 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:30.494800 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d874a-a19a-4423-94d9-8ce46b893543 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:30.495453 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d874a-a19a-4423-94d9-8ce46b893543 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:30.497738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 129/250] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:30.503602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf156c3-5050-417b-9488-6d731d6a4a56 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:30.504639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf156c3-5050-417b-9488-6d731d6a4a56 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:30.506254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf156c3-5050-417b-9488-6d731d6a4a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:30.548364 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-6bf156c3-5050-417b-9488-6d731d6a4a56 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:30.551509 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-6bf156c3-5050-417b-9488-6d731d6a4a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:30.582908 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 122/251] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 10 19:57:30.583768 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34138786-5b36-495a-9e8e-55edce2e6246 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:30.677305 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 130/252] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:30.678919 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf156c3-5050-417b-9488-6d731d6a4a56 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:30.768800 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34138786-5b36-495a-9e8e-55edce2e6246 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:30.778813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf156c3-5050-417b-9488-6d731d6a4a56 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:30.813188 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683b351-dc58-47db-a29b-628c40aae389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:30.813908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683b351-dc58-47db-a29b-628c40aae389 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:30.814292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683b351-dc58-47db-a29b-628c40aae389 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:30.814565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683b351-dc58-47db-a29b-628c40aae389 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:30.825364 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683b351-dc58-47db-a29b-628c40aae389 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:30.832976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-9683b351-dc58-47db-a29b-628c40aae389 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:30.832976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:30.832976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:30.832976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:30.832976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:30.832976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:30.832976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:30.832976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:30.832976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:30.832976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:30.832976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:30.832976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:30.832976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:30.832976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:30.832976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:30.832976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:30.833875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:30.833875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:30.833875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:30.833875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:30.833875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:30.833875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:30.833875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:30.833875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:30.833875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:30.833875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:30.833875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:30.833875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:30.833875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:30.833875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:30.833875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:30.833875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:30.834772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:30.834772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:30.834772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:30.834772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:30.834772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:30.834772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:30.834772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:30.834772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:30.834772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:30.834772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:30.834772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:30.834772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:30.834772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 131/253] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:57:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:30.835447 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683b351-dc58-47db-a29b-628c40aae389 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:30.863407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9f950-68e6-4c43-85af-06cd47e7e580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:30.864296 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9f950-68e6-4c43-85af-06cd47e7e580 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:30.864632 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9f950-68e6-4c43-85af-06cd47e7e580 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:30.864968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9f950-68e6-4c43-85af-06cd47e7e580 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:30.891008 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9f950-68e6-4c43-85af-06cd47e7e580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:30.906786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-88e9f950-68e6-4c43-85af-06cd47e7e580 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:30.906786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:30.906786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:30.906786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:30.906786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:30.906786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:30.906786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:30.906786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:30.906786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:30.906786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:30.906786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:30.906786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:30.906786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:30.906786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:30.906786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:30.906786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:30.907703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:30.907703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:30.907703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:30.907703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:30.907703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:30.907703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:30.907703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:30.907703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:30.907703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:30.907703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:30.907703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:30.907703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:30.907703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:30.907703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:30.907703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:30.907703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:30.908497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:30.908497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:30.908497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:30.908497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:30.908497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:30.908497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:30.908497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:30.908497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:30.908497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:30.908497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:30.908497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:30.908497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:30.908497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 123/254] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:30.909140 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9f950-68e6-4c43-85af-06cd47e7e580 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:30.954075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683b351-dc58-47db-a29b-628c40aae389 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:30.991899 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b85a0-efad-43c6-8b2d-410c7f9f78eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:30.992962 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b85a0-efad-43c6-8b2d-410c7f9f78eb None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:30.993247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b85a0-efad-43c6-8b2d-410c7f9f78eb None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:30.993523 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b85a0-efad-43c6-8b2d-410c7f9f78eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:31.005361 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b85a0-efad-43c6-8b2d-410c7f9f78eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:31.014462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-d50b85a0-efad-43c6-8b2d-410c7f9f78eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:31.014462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:31.014462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:31.014462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:31.014462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:31.014462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:31.014462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:31.014462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:31.014462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:31.014462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:31.014462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:31.014462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:31.014462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:31.014462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:31.014462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:31.014462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:31.015771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:31.015771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:31.015771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:31.015771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:31.015771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:31.015771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:31.015771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:31.015771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:31.015771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:31.015771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:31.015771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:31.015771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:31.015771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:31.015771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:31.015771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:31.015771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:31.017476 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:31.017476 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:31.017476 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:31.017476 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:31.017476 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:31.017476 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:31.017476 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:31.017476 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:31.017476 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:31.017476 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:31.017476 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:31.017476 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:31.017476 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 132/255] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:57:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:31.017476 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b85a0-efad-43c6-8b2d-410c7f9f78eb None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:31.095092 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9f950-68e6-4c43-85af-06cd47e7e580 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:31.131994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7c937-e29c-4c02-ae34-6429805735da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:31.133435 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7c937-e29c-4c02-ae34-6429805735da None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:31.133722 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7c937-e29c-4c02-ae34-6429805735da None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:31.134221 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7c937-e29c-4c02-ae34-6429805735da None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:31.134980 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7c937-e29c-4c02-ae34-6429805735da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:31.135279 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7c937-e29c-4c02-ae34-6429805735da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:31.136668 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7c937-e29c-4c02-ae34-6429805735da None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:31.154164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 124/256] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:31.155007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7c937-e29c-4c02-ae34-6429805735da None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:31.202021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b85a0-efad-43c6-8b2d-410c7f9f78eb None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:31.236569 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f82a11-b470-4cd4-a65b-c5ffd5593fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:31.237240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f82a11-b470-4cd4-a65b-c5ffd5593fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:31.237524 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f82a11-b470-4cd4-a65b-c5ffd5593fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:31.237803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f82a11-b470-4cd4-a65b-c5ffd5593fe4 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:31.238611 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f82a11-b470-4cd4-a65b-c5ffd5593fe4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:31.238925 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f82a11-b470-4cd4-a65b-c5ffd5593fe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:31.240341 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f82a11-b470-4cd4-a65b-c5ffd5593fe4 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:31.249612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 133/257] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:31.254519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f82a11-b470-4cd4-a65b-c5ffd5593fe4 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:31.260161 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7c937-e29c-4c02-ae34-6429805735da None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:31.301905 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e16a4a-0b96-4d35-a7b6-67e3cf961c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:31.303247 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e16a4a-0b96-4d35-a7b6-67e3cf961c26 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:31.303575 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e16a4a-0b96-4d35-a7b6-67e3cf961c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:31.303847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e16a4a-0b96-4d35-a7b6-67e3cf961c26 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:31.304601 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e16a4a-0b96-4d35-a7b6-67e3cf961c26 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:31.304899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e16a4a-0b96-4d35-a7b6-67e3cf961c26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:31.306367 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e16a4a-0b96-4d35-a7b6-67e3cf961c26 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:31.323852 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 125/258] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:57:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:31.324833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e16a4a-0b96-4d35-a7b6-67e3cf961c26 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:31.358967 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f82a11-b470-4cd4-a65b-c5ffd5593fe4 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:31.403916 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726bfe6-2856-4bda-bebe-e11dbb2e697f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:31.404806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726bfe6-2856-4bda-bebe-e11dbb2e697f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:31.405095 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726bfe6-2856-4bda-bebe-e11dbb2e697f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:31.405366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726bfe6-2856-4bda-bebe-e11dbb2e697f None admin] PATH_INFO: `/v3/users/a8217c0bdd0a4a7a8433de4d6410960f` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:31.406783 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726bfe6-2856-4bda-bebe-e11dbb2e697f None admin] RBAC: Authorizing `identity:get_user(user_id=a8217c0bdd0a4a7a8433de4d6410960f)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:31.408259 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726bfe6-2856-4bda-bebe-e11dbb2e697f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:31.410057 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 134/259] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:31 2022] GET /identity/v3/users/a8217c0bdd0a4a7a8433de4d6410960f => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:31.418920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726bfe6-2856-4bda-bebe-e11dbb2e697f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:31.451223 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e16a4a-0b96-4d35-a7b6-67e3cf961c26 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:31.491981 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba184ca-3fe1-48e7-9c45-a1f241046f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:31.493288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba184ca-3fe1-48e7-9c45-a1f241046f3c None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:31.493568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba184ca-3fe1-48e7-9c45-a1f241046f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:31.493892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba184ca-3fe1-48e7-9c45-a1f241046f3c None admin] PATH_INFO: `/v3/users/f37c0defef304bc69cc6edbfc44d6399` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:31.495188 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba184ca-3fe1-48e7-9c45-a1f241046f3c None admin] RBAC: Authorizing `identity:get_user(user_id=f37c0defef304bc69cc6edbfc44d6399)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:31.496766 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba184ca-3fe1-48e7-9c45-a1f241046f3c None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:31.498548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 126/260] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:31 2022] GET /identity/v3/users/f37c0defef304bc69cc6edbfc44d6399 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:31.499311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba184ca-3fe1-48e7-9c45-a1f241046f3c None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:31.547295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726bfe6-2856-4bda-bebe-e11dbb2e697f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:31.589512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2d75f-8559-4414-9a2d-84fabc98feb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:31.590414 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2d75f-8559-4414-9a2d-84fabc98feb9 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:31.590702 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2d75f-8559-4414-9a2d-84fabc98feb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:31.590981 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2d75f-8559-4414-9a2d-84fabc98feb9 None admin] PATH_INFO: `/v3/users/a8217c0bdd0a4a7a8433de4d6410960f` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:31.592184 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2d75f-8559-4414-9a2d-84fabc98feb9 None admin] RBAC: Authorizing `identity:get_user(user_id=a8217c0bdd0a4a7a8433de4d6410960f)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:31.593639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2d75f-8559-4414-9a2d-84fabc98feb9 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:31.595611 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 135/261] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:31 2022] GET /identity/v3/users/a8217c0bdd0a4a7a8433de4d6410960f => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:31.596395 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2d75f-8559-4414-9a2d-84fabc98feb9 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:31.634097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba184ca-3fe1-48e7-9c45-a1f241046f3c None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:31.676165 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656acc7-3675-4cd4-b8d6-37c24c4a2389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:31.676856 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656acc7-3675-4cd4-b8d6-37c24c4a2389 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:31.677143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656acc7-3675-4cd4-b8d6-37c24c4a2389 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:31.677980 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656acc7-3675-4cd4-b8d6-37c24c4a2389 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:31.727394 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2d75f-8559-4414-9a2d-84fabc98feb9 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:31.767997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6a8f8-cb4e-4ede-89ad-5706cfb242ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:31.769221 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6a8f8-cb4e-4ede-89ad-5706cfb242ea None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:31.769697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6a8f8-cb4e-4ede-89ad-5706cfb242ea None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:31.770186 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6a8f8-cb4e-4ede-89ad-5706cfb242ea None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:31.772927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6a8f8-cb4e-4ede-89ad-5706cfb242ea None admin] RBAC: Authorizing `identity:get_project(project_id=775bb0dd255b49ca81b1452cbe667589)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:31.776658 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6a8f8-cb4e-4ede-89ad-5706cfb242ea None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:31.782011 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 136/262] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:31 2022] GET /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:31.782999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6a8f8-cb4e-4ede-89ad-5706cfb242ea None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:31.814263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656acc7-3675-4cd4-b8d6-37c24c4a2389 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:31.816352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656acc7-3675-4cd4-b8d6-37c24c4a2389 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:31.956423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6a8f8-cb4e-4ede-89ad-5706cfb242ea None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:31.983619 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 127/263] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:31.984783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656acc7-3675-4cd4-b8d6-37c24c4a2389 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:32.034771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d00d9-0c0b-4ecd-9994-779503e2c39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:32.036102 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d00d9-0c0b-4ecd-9994-779503e2c39c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:32.036581 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d00d9-0c0b-4ecd-9994-779503e2c39c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:32.037101 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d00d9-0c0b-4ecd-9994-779503e2c39c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:32.114725 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d00d9-0c0b-4ecd-9994-779503e2c39c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:32.116850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d00d9-0c0b-4ecd-9994-779503e2c39c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:32.196214 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656acc7-3675-4cd4-b8d6-37c24c4a2389 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:32.224095 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 137/264] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:32.224897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d00d9-0c0b-4ecd-9994-779503e2c39c None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:32.284091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed74fc1-ebba-4175-8e3d-1773a991b8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:32.284996 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed74fc1-ebba-4175-8e3d-1773a991b8c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:32.285275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed74fc1-ebba-4175-8e3d-1773a991b8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:32.285549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed74fc1-ebba-4175-8e3d-1773a991b8c0 None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589/users/f37c0defef304bc69cc6edbfc44d6399/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:32.291415 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed74fc1-ebba-4175-8e3d-1773a991b8c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=775bb0dd255b49ca81b1452cbe667589, user_id=f37c0defef304bc69cc6edbfc44d6399, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:32.292872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed74fc1-ebba-4175-8e3d-1773a991b8c0 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:32.329577 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d00d9-0c0b-4ecd-9994-779503e2c39c None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:32.364086 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fc73f-e44d-4e71-8449-07daecd5c194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:32.365496 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fc73f-e44d-4e71-8449-07daecd5c194 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:32.365779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fc73f-e44d-4e71-8449-07daecd5c194 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:32.366053 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fc73f-e44d-4e71-8449-07daecd5c194 None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:32.367557 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fc73f-e44d-4e71-8449-07daecd5c194 None admin] RBAC: Authorizing `identity:get_project(project_id=775bb0dd255b49ca81b1452cbe667589)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:32.368971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fc73f-e44d-4e71-8449-07daecd5c194 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:32.371059 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 138/265] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:32 2022] GET /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:32.371830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fc73f-e44d-4e71-8449-07daecd5c194 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:32.382992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 128/266] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:57:31 2022] PUT /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589/users/f37c0defef304bc69cc6edbfc44d6399/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:57:32.383886 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed74fc1-ebba-4175-8e3d-1773a991b8c0 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:32.478224 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fc73f-e44d-4e71-8449-07daecd5c194 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:32.516753 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49e5b7-8064-4896-9a55-894b6e9b41f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:32.517648 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49e5b7-8064-4896-9a55-894b6e9b41f9 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:32.517927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49e5b7-8064-4896-9a55-894b6e9b41f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:32.518251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49e5b7-8064-4896-9a55-894b6e9b41f9 None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:32.521054 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec49e5b7-8064-4896-9a55-894b6e9b41f9 None admin] RBAC: Authorizing `identity:get_project(project_id=775bb0dd255b49ca81b1452cbe667589)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:32.522605 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec49e5b7-8064-4896-9a55-894b6e9b41f9 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:32.524446 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 139/267] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:32 2022] GET /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:32.530746 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49e5b7-8064-4896-9a55-894b6e9b41f9 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:32.632189 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed74fc1-ebba-4175-8e3d-1773a991b8c0 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:32.658178 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49e5b7-8064-4896-9a55-894b6e9b41f9 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:32.696240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee45d00-7eaf-4fd9-b531-9f12ffee66a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:32.696915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee45d00-7eaf-4fd9-b531-9f12ffee66a8 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:32.697196 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee45d00-7eaf-4fd9-b531-9f12ffee66a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:32.697468 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee45d00-7eaf-4fd9-b531-9f12ffee66a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:32.698369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee45d00-7eaf-4fd9-b531-9f12ffee66a8 None admin] RBAC: Adding query filter params (role.id=4fbb0a346f5f4f36bb32664ee799db42, scope.project.id=775bb0dd255b49ca81b1452cbe667589, user.id=a8217c0bdd0a4a7a8433de4d6410960f) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:32.698679 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee45d00-7eaf-4fd9-b531-9f12ffee66a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:32.700135 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee45d00-7eaf-4fd9-b531-9f12ffee66a8 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:32.706912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 140/268] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:57:32 2022] GET /identity/v3/role_assignments?user.id=a8217c0bdd0a4a7a8433de4d6410960f&scope.project.id=775bb0dd255b49ca81b1452cbe667589&role.id=4fbb0a346f5f4f36bb32664ee799db42 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:32.728140 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf0ce7-876c-43e3-a687-9fcb58aa7fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:32.728901 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf0ce7-876c-43e3-a687-9fcb58aa7fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:32.729124 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf0ce7-876c-43e3-a687-9fcb58aa7fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:32.729396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf0ce7-876c-43e3-a687-9fcb58aa7fa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:32.730322 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf0ce7-876c-43e3-a687-9fcb58aa7fa2 None admin] RBAC: Adding query filter params (role.id=5c87e804d0764309ba08ad6e06b23a87, scope.project.id=775bb0dd255b49ca81b1452cbe667589, user.id=a8217c0bdd0a4a7a8433de4d6410960f) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:32.730628 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf0ce7-876c-43e3-a687-9fcb58aa7fa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:32.732134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf0ce7-876c-43e3-a687-9fcb58aa7fa2 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:32.739871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 129/269] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:57:32 2022] GET /identity/v3/role_assignments?user.id=a8217c0bdd0a4a7a8433de4d6410960f&scope.project.id=775bb0dd255b49ca81b1452cbe667589&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 608 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:32.763731 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00218d-6d6a-4b54-bfa9-4b6afa1aae19 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:32.764041 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00218d-6d6a-4b54-bfa9-4b6afa1aae19 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:32.764290 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00218d-6d6a-4b54-bfa9-4b6afa1aae19 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:32.765892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 141/270] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:32.780666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fc0a5-d0db-4ba1-a90b-37d8f3e6f3ef None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:32.780989 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fc0a5-d0db-4ba1-a90b-37d8f3e6f3ef None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:32.781343 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fc0a5-d0db-4ba1-a90b-37d8f3e6f3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:32.800145 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-455fc0a5-d0db-4ba1-a90b-37d8f3e6f3ef None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:32.801373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-455fc0a5-d0db-4ba1-a90b-37d8f3e6f3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:32.878171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 130/271] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:32.920665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00218d-6d6a-4b54-bfa9-4b6afa1aae19 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:32.926636 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497042b2-530b-4cc5-9fd5-bbe0135b898a None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:32.926916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497042b2-530b-4cc5-9fd5-bbe0135b898a None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:32.927198 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497042b2-530b-4cc5-9fd5-bbe0135b898a None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:32.928979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 131/272] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:32.935457 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a93623-8ee4-4971-8229-f295b891122c None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:32.935741 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a93623-8ee4-4971-8229-f295b891122c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:32.936024 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a93623-8ee4-4971-8229-f295b891122c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:32.958028 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-a2a93623-8ee4-4971-8229-f295b891122c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:32.959810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a2a93623-8ee4-4971-8229-f295b891122c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:33.046254 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 132/273] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:57:33.077512 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a93623-8ee4-4971-8229-f295b891122c None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:33.094404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00218d-6d6a-4b54-bfa9-4b6afa1aae19 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:33.136554 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf89df-31e1-4684-8b21-23c3363d3e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:33.137633 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf89df-31e1-4684-8b21-23c3363d3e6b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:33.137914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf89df-31e1-4684-8b21-23c3363d3e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:33.138383 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf89df-31e1-4684-8b21-23c3363d3e6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:33.151485 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf89df-31e1-4684-8b21-23c3363d3e6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:33.161642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-90cf89df-31e1-4684-8b21-23c3363d3e6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:33.161642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:33.161642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:33.161642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:33.161642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:33.161642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:33.161642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:33.161642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:33.161642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:33.161642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:33.161642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:33.161642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:33.161642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:33.161642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:33.161642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:33.161642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:33.162585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:33.162585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:33.162585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:33.162585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:33.162585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:33.162585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:33.162585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:33.162585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:33.162585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:33.162585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:33.162585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:33.162585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:33.162585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:33.162585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:33.162585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:33.162585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:33.163386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:33.163386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:33.163386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:33.163386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:33.163386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:33.163386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:33.163386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:33.163386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:33.163386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:33.163386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:33.163386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:33.163386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:33.163983 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 142/274] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:33.170055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf89df-31e1-4684-8b21-23c3363d3e6b None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:33.209397 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a93623-8ee4-4971-8229-f295b891122c None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:33.251652 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c3cec-e697-4713-84c3-7b1be9fea554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:33.256007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c3cec-e697-4713-84c3-7b1be9fea554 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:33.256007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c3cec-e697-4713-84c3-7b1be9fea554 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:33.256007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c3cec-e697-4713-84c3-7b1be9fea554 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:33.256007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c3cec-e697-4713-84c3-7b1be9fea554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:33.257257 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c3cec-e697-4713-84c3-7b1be9fea554 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:33.259485 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 133/275] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:57:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:33.264874 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c3cec-e697-4713-84c3-7b1be9fea554 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:33.300661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf89df-31e1-4684-8b21-23c3363d3e6b None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:33.348976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f0b8b-e950-490a-b1ee-652605d59aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:33.349677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f0b8b-e950-490a-b1ee-652605d59aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:33.349962 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f0b8b-e950-490a-b1ee-652605d59aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:33.350273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f0b8b-e950-490a-b1ee-652605d59aa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:33.351034 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f0b8b-e950-490a-b1ee-652605d59aa8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:33.351369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f0b8b-e950-490a-b1ee-652605d59aa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:33.352725 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f0b8b-e950-490a-b1ee-652605d59aa8 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:33.364591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 143/276] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:33.370210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f0b8b-e950-490a-b1ee-652605d59aa8 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:33.409961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c3cec-e697-4713-84c3-7b1be9fea554 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:33.453482 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ade69-40f5-4d7e-9885-b0fbac06090f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:33.454553 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ade69-40f5-4d7e-9885-b0fbac06090f None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:33.454839 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ade69-40f5-4d7e-9885-b0fbac06090f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:33.455115 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ade69-40f5-4d7e-9885-b0fbac06090f None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:33.455836 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ade69-40f5-4d7e-9885-b0fbac06090f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:33.457283 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ade69-40f5-4d7e-9885-b0fbac06090f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:33.479473 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f0b8b-e950-490a-b1ee-652605d59aa8 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:33.512052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908953d-1e55-4653-81c0-004a18c9a2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:33.513039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908953d-1e55-4653-81c0-004a18c9a2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:33.513330 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908953d-1e55-4653-81c0-004a18c9a2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:33.513617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908953d-1e55-4653-81c0-004a18c9a2e3 None admin] PATH_INFO: `/v3/users/07313b26b7d8455ab4d49b7ea57a1956` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:33.526414 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908953d-1e55-4653-81c0-004a18c9a2e3 None admin] RBAC: Authorizing `identity:get_user(user_id=07313b26b7d8455ab4d49b7ea57a1956)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:33.527857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908953d-1e55-4653-81c0-004a18c9a2e3 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:33.529513 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 144/277] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:33 2022] GET /identity/v3/users/07313b26b7d8455ab4d49b7ea57a1956 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:33.531621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965009ef-6f7b-440b-b51d-c85c18dc366c None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:33.531961 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965009ef-6f7b-440b-b51d-c85c18dc366c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:33.532209 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965009ef-6f7b-440b-b51d-c85c18dc366c None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:33.533596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 145/278] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:33.538380 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965009ef-6f7b-440b-b51d-c85c18dc366c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:33.553961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 134/279] 10.222.0.65 () {60 vars in 1132 bytes} [Tue May 10 19:57:33 2022] POST /identity/v3/users => generated 293 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 19:57:33.555843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030937f-aa89-4093-908a-67b54240c4b5 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:33.556284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030937f-aa89-4093-908a-67b54240c4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:33.556688 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030937f-aa89-4093-908a-67b54240c4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:33.575419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-c030937f-aa89-4093-908a-67b54240c4b5 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:33.577144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c030937f-aa89-4093-908a-67b54240c4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:33.674753 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 135/280] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:33.676800 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965009ef-6f7b-440b-b51d-c85c18dc366c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:33.691561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030937f-aa89-4093-908a-67b54240c4b5 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:33.720890 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0b690-17e4-4036-bf61-3c475889ea0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:33.721878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0b690-17e4-4036-bf61-3c475889ea0a None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:33.722332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0b690-17e4-4036-bf61-3c475889ea0a None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:33.722669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0b690-17e4-4036-bf61-3c475889ea0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:33.723761 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0b690-17e4-4036-bf61-3c475889ea0a None admin] RBAC: Adding query filter params (role.id=5f62ce25636443f8a5add67f208afe62, user.id=07313b26b7d8455ab4d49b7ea57a1956, scope.system=all) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:33.724169 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0b690-17e4-4036-bf61-3c475889ea0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:33.726576 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0b690-17e4-4036-bf61-3c475889ea0a None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:33.733660 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 146/281] 10.222.0.65 () {56 vars in 1308 bytes} [Tue May 10 19:57:33 2022] GET /identity/v3/role_assignments?user.id=07313b26b7d8455ab4d49b7ea57a1956&scope.system=all&role.id=5f62ce25636443f8a5add67f208afe62 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:33.807051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030937f-aa89-4093-908a-67b54240c4b5 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:33.844709 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707cb4f-55f2-4ba4-ae58-461a0149dbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:33.845519 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707cb4f-55f2-4ba4-ae58-461a0149dbe3 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:33.845799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707cb4f-55f2-4ba4-ae58-461a0149dbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:33.846071 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707cb4f-55f2-4ba4-ae58-461a0149dbe3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:33.858442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707cb4f-55f2-4ba4-ae58-461a0149dbe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:33.866941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-b707cb4f-55f2-4ba4-ae58-461a0149dbe3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 19:57:33.866941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:33.866941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:33.866941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:33.866941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:33.866941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:33.866941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:33.866941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:33.866941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:33.866941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:33.866941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:33.866941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:33.866941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:33.866941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:33.866941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:33.866941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:33.868160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:33.868160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:33.868160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:33.868160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:33.868160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:33.868160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:33.868160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:33.868160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:33.868160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:33.868160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:33.868160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:33.868160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:33.868160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:33.868160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:33.868160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:33.868160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:33.869210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:33.869210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:33.869210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:33.869210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:33.869210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:33.869210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:33.869210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:33.869210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:33.869210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:33.869210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:33.869210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 19:57:33.869210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:33.869210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 136/282] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:57:33.876533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0b690-17e4-4036-bf61-3c475889ea0a None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:33.998333 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0b690-17e4-4036-bf61-3c475889ea0a None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:34.042846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86290729-f174-4850-ac91-fd68ca1e7268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:34.043542 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86290729-f174-4850-ac91-fd68ca1e7268 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:34.043830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86290729-f174-4850-ac91-fd68ca1e7268 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:34.044109 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86290729-f174-4850-ac91-fd68ca1e7268 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:34.044881 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86290729-f174-4850-ac91-fd68ca1e7268 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:34.045193 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86290729-f174-4850-ac91-fd68ca1e7268 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:34.046660 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86290729-f174-4850-ac91-fd68ca1e7268 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:34.056399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 147/283] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:34.062435 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707cb4f-55f2-4ba4-ae58-461a0149dbe3 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:34.209038 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707cb4f-55f2-4ba4-ae58-461a0149dbe3 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:34.249715 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e7066-3040-40c2-a6ca-5414006c5eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:34.250636 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e7066-3040-40c2-a6ca-5414006c5eaf None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:34.251735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e7066-3040-40c2-a6ca-5414006c5eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:34.251735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e7066-3040-40c2-a6ca-5414006c5eaf None admin] PATH_INFO: `/v3/users/6ccfbf763b834334b259636e317cb678` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:34.252395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e7066-3040-40c2-a6ca-5414006c5eaf None admin] RBAC: Authorizing `identity:get_user(user_id=6ccfbf763b834334b259636e317cb678)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:34.253863 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e7066-3040-40c2-a6ca-5414006c5eaf None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:34.255738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 137/284] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:34 2022] GET /identity/v3/users/6ccfbf763b834334b259636e317cb678 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:34.261070 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86290729-f174-4850-ac91-fd68ca1e7268 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:34.380945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86290729-f174-4850-ac91-fd68ca1e7268 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:34.423375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310f933-6da8-4c01-851a-a09a3e07c88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:34.424257 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310f933-6da8-4c01-851a-a09a3e07c88d None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:34.424543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310f933-6da8-4c01-851a-a09a3e07c88d None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:34.424821 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310f933-6da8-4c01-851a-a09a3e07c88d None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:34.426193 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310f933-6da8-4c01-851a-a09a3e07c88d None admin] RBAC: Authorizing `identity:get_project(project_id=775bb0dd255b49ca81b1452cbe667589)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:34.428175 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310f933-6da8-4c01-851a-a09a3e07c88d None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:34.429338 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 148/285] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:34 2022] GET /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:34.436236 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e7066-3040-40c2-a6ca-5414006c5eaf None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:34.558069 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e7066-3040-40c2-a6ca-5414006c5eaf None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:34.602779 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a188b-055c-469d-b2a4-f8db759cbad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:34.604011 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a188b-055c-469d-b2a4-f8db759cbad1 None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:34.604526 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a188b-055c-469d-b2a4-f8db759cbad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:34.605012 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a188b-055c-469d-b2a4-f8db759cbad1 None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589/users/6ccfbf763b834334b259636e317cb678/roles/f7a4054724ca4163960cf635d13b055f` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:34.609000 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a188b-055c-469d-b2a4-f8db759cbad1 None admin] RBAC: Authorizing `identity:create_grant(project_id=775bb0dd255b49ca81b1452cbe667589, user_id=6ccfbf763b834334b259636e317cb678, role_id=f7a4054724ca4163960cf635d13b055f)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:34.610892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a188b-055c-469d-b2a4-f8db759cbad1 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:34.705512 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 138/286] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:57:34 2022] PUT /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589/users/6ccfbf763b834334b259636e317cb678/roles/f7a4054724ca4163960cf635d13b055f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:57:35.413313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697403ff-0d8f-49d7-9f40-ae6ce1e4d674 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:35.413829 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697403ff-0d8f-49d7-9f40-ae6ce1e4d674 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:35.413904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697403ff-0d8f-49d7-9f40-ae6ce1e4d674 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:35.415565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 149/287] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:35.420504 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dea92d-2b74-4631-b767-2b41a67d9f4a None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:35.420907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dea92d-2b74-4631-b767-2b41a67d9f4a None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:35.421117 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dea92d-2b74-4631-b767-2b41a67d9f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:35.443073 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-07dea92d-2b74-4631-b767-2b41a67d9f4a None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:35.444408 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-07dea92d-2b74-4631-b767-2b41a67d9f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:35.528389 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 139/288] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:35.564108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697403ff-0d8f-49d7-9f40-ae6ce1e4d674 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:35.687827 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697403ff-0d8f-49d7-9f40-ae6ce1e4d674 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:35.731264 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d571dc-34b3-4641-9ce0-43d26e5aff19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:35.732065 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d571dc-34b3-4641-9ce0-43d26e5aff19 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:35.732353 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d571dc-34b3-4641-9ce0-43d26e5aff19 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:35.732636 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d571dc-34b3-4641-9ce0-43d26e5aff19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:35.747138 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d571dc-34b3-4641-9ce0-43d26e5aff19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:35.754997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-b1d571dc-34b3-4641-9ce0-43d26e5aff19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:35.754997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:35.754997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:35.754997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:35.754997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:35.754997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:35.754997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:35.754997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:35.754997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:35.754997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:35.754997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:35.754997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:35.754997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:35.754997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:35.754997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:35.754997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:35.755912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:35.755912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:35.755912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:35.755912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:35.755912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:35.755912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:35.755912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:35.755912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:35.755912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:35.755912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:35.755912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:35.755912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:35.755912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:35.755912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:35.755912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:35.755912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:35.757137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:35.757137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:35.757137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:35.757137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:35.757137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:35.757137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:35.757137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:35.757137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:35.757137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:35.757137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:35.757137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:35.757137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:35.757137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 150/289] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:35.762011 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dea92d-2b74-4631-b767-2b41a67d9f4a None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:35.863509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dea92d-2b74-4631-b767-2b41a67d9f4a None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:35.905262 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e0a1d-206a-4cba-b7b0-7cc15aeef488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:35.905943 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e0a1d-206a-4cba-b7b0-7cc15aeef488 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:35.906254 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e0a1d-206a-4cba-b7b0-7cc15aeef488 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:35.906533 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e0a1d-206a-4cba-b7b0-7cc15aeef488 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:35.907297 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e0a1d-206a-4cba-b7b0-7cc15aeef488 None admin] RBAC: Adding query filter params (name=member) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:35.907605 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e0a1d-206a-4cba-b7b0-7cc15aeef488 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:35.908995 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e0a1d-206a-4cba-b7b0-7cc15aeef488 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:35.918456 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 140/290] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:35.924028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d571dc-34b3-4641-9ce0-43d26e5aff19 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:36.027307 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d571dc-34b3-4641-9ce0-43d26e5aff19 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:36.065546 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bd371-ae10-4a4d-831f-5f0701e072c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:36.066454 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bd371-ae10-4a4d-831f-5f0701e072c0 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:36.066742 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bd371-ae10-4a4d-831f-5f0701e072c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:36.067017 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bd371-ae10-4a4d-831f-5f0701e072c0 None admin] PATH_INFO: `/v3/users/f37c0defef304bc69cc6edbfc44d6399` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:36.068283 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1bd371-ae10-4a4d-831f-5f0701e072c0 None admin] RBAC: Authorizing `identity:get_user(user_id=f37c0defef304bc69cc6edbfc44d6399)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:36.069722 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1bd371-ae10-4a4d-831f-5f0701e072c0 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:36.071385 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 151/291] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:35 2022] GET /identity/v3/users/f37c0defef304bc69cc6edbfc44d6399 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:36.076913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e0a1d-206a-4cba-b7b0-7cc15aeef488 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:36.194666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e0a1d-206a-4cba-b7b0-7cc15aeef488 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:36.237378 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ad1aa-f945-4b63-87ed-9aab92bebfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:36.240592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ad1aa-f945-4b63-87ed-9aab92bebfa9 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:36.240882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ad1aa-f945-4b63-87ed-9aab92bebfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:36.241179 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ad1aa-f945-4b63-87ed-9aab92bebfa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:36.331679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ad1aa-f945-4b63-87ed-9aab92bebfa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:36.333101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ad1aa-f945-4b63-87ed-9aab92bebfa9 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:36.434667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 141/292] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:36.440854 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bd371-ae10-4a4d-831f-5f0701e072c0 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:36.503706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997b382-05d8-4e8e-afae-5386b18a11b0 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:36.504227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997b382-05d8-4e8e-afae-5386b18a11b0 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:36.504649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997b382-05d8-4e8e-afae-5386b18a11b0 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:36.506750 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 142/293] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:36.516917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ede2b-0b1d-4e17-abef-3c6023f31e1b None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:36.517204 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ede2b-0b1d-4e17-abef-3c6023f31e1b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:36.517479 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ede2b-0b1d-4e17-abef-3c6023f31e1b None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:36.518832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 143/294] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:57:36.519858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba936e4-74bb-4cae-ae39-6139f3222606 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:36.520135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba936e4-74bb-4cae-ae39-6139f3222606 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:36.520408 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba936e4-74bb-4cae-ae39-6139f3222606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:36.540566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-cba936e4-74bb-4cae-ae39-6139f3222606 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:36.541740 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-cba936e4-74bb-4cae-ae39-6139f3222606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:36.574145 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bd371-ae10-4a4d-831f-5f0701e072c0 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:36.625983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 144/295] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:36.626640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657d8c7-dbb6-4652-b92a-60c186f4eef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:36.627038 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a971c7-8a91-4044-9247-3c871a5d0186 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:36.627324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a971c7-8a91-4044-9247-3c871a5d0186 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:36.627596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a971c7-8a91-4044-9247-3c871a5d0186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:36.628150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657d8c7-dbb6-4652-b92a-60c186f4eef5 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:36.628591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657d8c7-dbb6-4652-b92a-60c186f4eef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:36.629010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657d8c7-dbb6-4652-b92a-60c186f4eef5 None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:36.631333 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c657d8c7-dbb6-4652-b92a-60c186f4eef5 None admin] RBAC: Authorizing `identity:get_project(project_id=775bb0dd255b49ca81b1452cbe667589)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:36.633269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c657d8c7-dbb6-4652-b92a-60c186f4eef5 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:36.636387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 152/296] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:36 2022] GET /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:36.643730 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657d8c7-dbb6-4652-b92a-60c186f4eef5 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:36.653792 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-56a971c7-8a91-4044-9247-3c871a5d0186 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:36.655013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-56a971c7-8a91-4044-9247-3c871a5d0186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:36.720724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 145/297] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:36.730307 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a971c7-8a91-4044-9247-3c871a5d0186 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:36.768535 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657d8c7-dbb6-4652-b92a-60c186f4eef5 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:36.800545 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f98e0a-59e1-47a3-84ac-4b76949a6410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:36.801200 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f98e0a-59e1-47a3-84ac-4b76949a6410 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:36.801494 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f98e0a-59e1-47a3-84ac-4b76949a6410 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:36.801768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f98e0a-59e1-47a3-84ac-4b76949a6410 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:36.803173 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f98e0a-59e1-47a3-84ac-4b76949a6410 None admin] RBAC: Adding query filter params (role.id=5f62ce25636443f8a5add67f208afe62, scope.project.id=775bb0dd255b49ca81b1452cbe667589, user.id=f37c0defef304bc69cc6edbfc44d6399) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:36.803498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f98e0a-59e1-47a3-84ac-4b76949a6410 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:36.804936 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f98e0a-59e1-47a3-84ac-4b76949a6410 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:36.810711 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 153/298] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:57:36 2022] GET /identity/v3/role_assignments?user.id=f37c0defef304bc69cc6edbfc44d6399&scope.project.id=775bb0dd255b49ca81b1452cbe667589&role.id=5f62ce25636443f8a5add67f208afe62 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:36.811923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a57c2-f0f4-4586-8525-3bc3ef1e84a6 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:36.812209 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a57c2-f0f4-4586-8525-3bc3ef1e84a6 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:36.812483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a57c2-f0f4-4586-8525-3bc3ef1e84a6 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:36.813866 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 154/299] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:36.814498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a57c2-f0f4-4586-8525-3bc3ef1e84a6 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:36.836820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a971c7-8a91-4044-9247-3c871a5d0186 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:36.877681 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd4fcc-93fa-4708-b51e-fd0f94729623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:36.878440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd4fcc-93fa-4708-b51e-fd0f94729623 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:36.878726 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd4fcc-93fa-4708-b51e-fd0f94729623 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:36.879005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd4fcc-93fa-4708-b51e-fd0f94729623 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:36.891170 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd4fcc-93fa-4708-b51e-fd0f94729623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:36.899608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-23cd4fcc-93fa-4708-b51e-fd0f94729623 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 19:57:36.899608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:36.899608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:36.899608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:36.899608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:36.899608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:36.899608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:36.899608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:36.899608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:36.899608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:36.899608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:36.899608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:36.899608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:36.899608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:36.899608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:36.899608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:36.900561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:36.900561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:36.900561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:36.900561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:36.900561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:36.900561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:36.900561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:36.900561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:36.900561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:36.900561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:36.900561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:36.900561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:36.900561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:36.900561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:36.900561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:36.900561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:36.901822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:36.901822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:36.901822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:36.901822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:36.901822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:36.901822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:36.901822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:36.901822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:36.901822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:36.901822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:36.901822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 19:57:36.901822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:36.907948 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 146/300] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:36.908125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fb14a-6b70-4662-83a0-d4b2eae0936b None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:36.908324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fb14a-6b70-4662-83a0-d4b2eae0936b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:36.908610 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fb14a-6b70-4662-83a0-d4b2eae0936b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:36.928764 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-498fb14a-6b70-4662-83a0-d4b2eae0936b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:36.929354 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-498fb14a-6b70-4662-83a0-d4b2eae0936b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:36.961083 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a57c2-f0f4-4586-8525-3bc3ef1e84a6 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:37.002031 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6c1d0-6013-44a0-b142-117c25c04898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:37.003630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6c1d0-6013-44a0-b142-117c25c04898 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:37.004113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6c1d0-6013-44a0-b142-117c25c04898 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:37.004584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6c1d0-6013-44a0-b142-117c25c04898 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:37.010352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 147/301] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:37.011279 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fb14a-6b70-4662-83a0-d4b2eae0936b None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:37.017131 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6c1d0-6013-44a0-b142-117c25c04898 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:37.026592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-8cb6c1d0-6013-44a0-b142-117c25c04898 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:37.026592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:37.026592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:37.026592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:37.026592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:37.026592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:37.026592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:37.026592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:37.026592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:37.026592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:37.026592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:37.026592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:37.026592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:37.026592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:37.026592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:37.026592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:37.027495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:37.027495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:37.027495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:37.027495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:37.027495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:37.027495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:37.027495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:37.027495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:37.027495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:37.027495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:37.027495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:37.027495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:37.027495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:37.027495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:37.027495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:37.027495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:37.028310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:37.028310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:37.028310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:37.028310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:37.028310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:37.028310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:37.028310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:37.028310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:37.028310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:37.028310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:37.028310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:37.028310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:37.031240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 155/302] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:37.032052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6c1d0-6013-44a0-b142-117c25c04898 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:37.138827 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fb14a-6b70-4662-83a0-d4b2eae0936b None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:37.143103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6c1d0-6013-44a0-b142-117c25c04898 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:37.172391 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a85e3-9e45-4e14-8674-27f425cd28f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:37.173060 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a85e3-9e45-4e14-8674-27f425cd28f6 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:37.173346 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a85e3-9e45-4e14-8674-27f425cd28f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:37.173624 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a85e3-9e45-4e14-8674-27f425cd28f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:37.174433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a85e3-9e45-4e14-8674-27f425cd28f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:37.174745 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a85e3-9e45-4e14-8674-27f425cd28f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:37.175744 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f315b-6329-453d-a53d-865f442cf047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:37.176131 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a85e3-9e45-4e14-8674-27f425cd28f6 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:37.176527 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f315b-6329-453d-a53d-865f442cf047 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:37.176808 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f315b-6329-453d-a53d-865f442cf047 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:37.177645 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f315b-6329-453d-a53d-865f442cf047 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:37.178829 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f315b-6329-453d-a53d-865f442cf047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:37.180243 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f315b-6329-453d-a53d-865f442cf047 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:37.181844 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 156/303] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:57:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:37.182632 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f315b-6329-453d-a53d-865f442cf047 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:37.193523 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 148/304] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:37.193523 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a85e3-9e45-4e14-8674-27f425cd28f6 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:37.283762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f315b-6329-453d-a53d-865f442cf047 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:37.297141 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a85e3-9e45-4e14-8674-27f425cd28f6 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:37.320286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5243627a-145c-4178-8732-8362a4d2cd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:37.320952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243627a-145c-4178-8732-8362a4d2cd8e None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:37.321237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243627a-145c-4178-8732-8362a4d2cd8e None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:37.321512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243627a-145c-4178-8732-8362a4d2cd8e None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:37.322293 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5243627a-145c-4178-8732-8362a4d2cd8e None admin] RBAC: Adding query filter params (name=member) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:37.322612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5243627a-145c-4178-8732-8362a4d2cd8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:37.323977 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5243627a-145c-4178-8732-8362a4d2cd8e None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:37.329932 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5726e-d86e-46da-bfbf-74c3cfcbf6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:37.330622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5726e-d86e-46da-bfbf-74c3cfcbf6a9 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:37.330908 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5726e-d86e-46da-bfbf-74c3cfcbf6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:37.331186 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5726e-d86e-46da-bfbf-74c3cfcbf6a9 None admin] PATH_INFO: `/v3/groups` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:37.331880 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5726e-d86e-46da-bfbf-74c3cfcbf6a9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:37.337323 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 157/305] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:37.337323 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5243627a-145c-4178-8732-8362a4d2cd8e None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:37.337522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5726e-d86e-46da-bfbf-74c3cfcbf6a9 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:37.415483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 149/306] 10.222.0.65 () {60 vars in 1134 bytes} [Tue May 10 19:57:37 2022] POST /identity/v3/groups => generated 229 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:57:37.415483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5726e-d86e-46da-bfbf-74c3cfcbf6a9 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:37.448998 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5243627a-145c-4178-8732-8362a4d2cd8e None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:37.485291 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e46d18-226b-4332-af15-2419b4eca377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:37.486202 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e46d18-226b-4332-af15-2419b4eca377 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:37.486487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e46d18-226b-4332-af15-2419b4eca377 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:37.486763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e46d18-226b-4332-af15-2419b4eca377 None admin] PATH_INFO: `/v3/users/f2f4ead7f8d24073bd668d70b83e10cc` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:37.501460 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e46d18-226b-4332-af15-2419b4eca377 None admin] RBAC: Authorizing `identity:get_user(user_id=f2f4ead7f8d24073bd668d70b83e10cc)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:37.502994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e46d18-226b-4332-af15-2419b4eca377 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:37.509776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 158/307] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:37 2022] GET /identity/v3/users/f2f4ead7f8d24073bd668d70b83e10cc => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:37.509776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bbe48-3377-4c8b-8295-aeba92d2fe40 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:37.509776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bbe48-3377-4c8b-8295-aeba92d2fe40 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:37.509776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bbe48-3377-4c8b-8295-aeba92d2fe40 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:37.509776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 159/308] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:37.513867 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bbe48-3377-4c8b-8295-aeba92d2fe40 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:37.522127 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5726e-d86e-46da-bfbf-74c3cfcbf6a9 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:37.556590 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf454f-a03e-407c-a126-81be7efbd2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:37.557475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf454f-a03e-407c-a126-81be7efbd2af None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:37.557759 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf454f-a03e-407c-a126-81be7efbd2af None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:37.558037 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf454f-a03e-407c-a126-81be7efbd2af None admin] PATH_INFO: `/v3/users/07313b26b7d8455ab4d49b7ea57a1956` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:37.559349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf454f-a03e-407c-a126-81be7efbd2af None admin] RBAC: Authorizing `identity:get_user(user_id=07313b26b7d8455ab4d49b7ea57a1956)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:37.560884 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf454f-a03e-407c-a126-81be7efbd2af None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:37.562585 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 150/309] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:37 2022] GET /identity/v3/users/07313b26b7d8455ab4d49b7ea57a1956 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:37.563913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a7a5f-5f24-4a71-b72d-b9e2825b8ee2 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:37.564233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a7a5f-5f24-4a71-b72d-b9e2825b8ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:37.564516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a7a5f-5f24-4a71-b72d-b9e2825b8ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:37.581888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-e54a7a5f-5f24-4a71-b72d-b9e2825b8ee2 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:37.583999 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-e54a7a5f-5f24-4a71-b72d-b9e2825b8ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:37.623141 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bbe48-3377-4c8b-8295-aeba92d2fe40 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:37.656782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 151/310] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:37.662100 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a7a5f-5f24-4a71-b72d-b9e2825b8ee2 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:37.663941 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b600b6-7c4c-4f32-8d5c-f64cfd6c20c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:37.664656 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b600b6-7c4c-4f32-8d5c-f64cfd6c20c7 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:37.664940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b600b6-7c4c-4f32-8d5c-f64cfd6c20c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:37.665260 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b600b6-7c4c-4f32-8d5c-f64cfd6c20c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:37.666274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b600b6-7c4c-4f32-8d5c-f64cfd6c20c7 None admin] RBAC: Adding query filter params (role.id=f7a4054724ca4163960cf635d13b055f, user.id=f2f4ead7f8d24073bd668d70b83e10cc, scope.system=all) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:37.666536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b600b6-7c4c-4f32-8d5c-f64cfd6c20c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:37.668069 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b600b6-7c4c-4f32-8d5c-f64cfd6c20c7 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:37.673537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 160/311] 10.222.0.65 () {56 vars in 1308 bytes} [Tue May 10 19:57:37 2022] GET /identity/v3/role_assignments?user.id=f2f4ead7f8d24073bd668d70b83e10cc&scope.system=all&role.id=f7a4054724ca4163960cf635d13b055f => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:37.682327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b600b6-7c4c-4f32-8d5c-f64cfd6c20c7 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:37.796562 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a7a5f-5f24-4a71-b72d-b9e2825b8ee2 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:37.798290 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b600b6-7c4c-4f32-8d5c-f64cfd6c20c7 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:37.835511 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54706e62-c756-4354-8b4b-aa27f1e54b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:37.836729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54706e62-c756-4354-8b4b-aa27f1e54b98 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:37.837211 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54706e62-c756-4354-8b4b-aa27f1e54b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:37.837674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54706e62-c756-4354-8b4b-aa27f1e54b98 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:37.842227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25b269-bd71-4e78-9e0a-f3691b2474ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:37.843114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25b269-bd71-4e78-9e0a-f3691b2474ae None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:37.843394 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25b269-bd71-4e78-9e0a-f3691b2474ae None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:37.843666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25b269-bd71-4e78-9e0a-f3691b2474ae None admin] PATH_INFO: `/v3/system/users/07313b26b7d8455ab4d49b7ea57a1956/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:37.845590 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25b269-bd71-4e78-9e0a-f3691b2474ae None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=07313b26b7d8455ab4d49b7ea57a1956, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:37.847041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25b269-bd71-4e78-9e0a-f3691b2474ae None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:37.851866 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54706e62-c756-4354-8b4b-aa27f1e54b98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:37.860327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-54706e62-c756-4354-8b4b-aa27f1e54b98 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 19:57:37.860327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:37.860327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:37.860327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:37.860327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:37.860327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:37.860327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:37.860327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:37.860327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:37.860327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:37.860327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:37.860327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:37.860327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:37.860327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:37.860327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:37.860327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:37.880907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:37.880907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:37.880907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:37.880907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:37.880907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:37.880907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:37.880907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:37.880907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:37.880907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:37.880907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:37.880907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:37.880907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:37.880907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:37.880907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:37.880907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:37.880907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:37.881718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:37.881718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:37.881718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:37.881718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:37.881718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:37.881718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:37.881718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:37.881718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:37.881718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:37.881718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:37.881718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 19:57:37.881718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:37.881718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 161/312] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:57:37.881718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54706e62-c756-4354-8b4b-aa27f1e54b98 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:37.956538 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 152/313] 10.222.0.65 () {58 vars in 1334 bytes} [Tue May 10 19:57:37 2022] PUT /identity/v3/system/users/07313b26b7d8455ab4d49b7ea57a1956/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:57:37.973638 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54706e62-c756-4354-8b4b-aa27f1e54b98 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:38.014093 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a8a2b-9738-4cb9-a3d5-26246ae428dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:38.014779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a8a2b-9738-4cb9-a3d5-26246ae428dc None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:38.015065 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a8a2b-9738-4cb9-a3d5-26246ae428dc None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:38.015344 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a8a2b-9738-4cb9-a3d5-26246ae428dc None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:38.016122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a8a2b-9738-4cb9-a3d5-26246ae428dc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:38.016437 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a8a2b-9738-4cb9-a3d5-26246ae428dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:38.018096 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a8a2b-9738-4cb9-a3d5-26246ae428dc None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:38.028784 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 162/314] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:38.034317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25b269-bd71-4e78-9e0a-f3691b2474ae None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:38.144223 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25b269-bd71-4e78-9e0a-f3691b2474ae None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:38.180493 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4e230-8ed4-43fd-ad98-0478add7f780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:38.181374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4e230-8ed4-43fd-ad98-0478add7f780 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:38.181660 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4e230-8ed4-43fd-ad98-0478add7f780 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:38.181936 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4e230-8ed4-43fd-ad98-0478add7f780 None admin] PATH_INFO: `/v3/users/6ccfbf763b834334b259636e317cb678` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:38.184051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4e230-8ed4-43fd-ad98-0478add7f780 None admin] RBAC: Authorizing `identity:get_user(user_id=6ccfbf763b834334b259636e317cb678)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:38.185242 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4e230-8ed4-43fd-ad98-0478add7f780 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:38.191265 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 153/315] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:38 2022] GET /identity/v3/users/6ccfbf763b834334b259636e317cb678 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:38.192460 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a8a2b-9738-4cb9-a3d5-26246ae428dc None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:38.300455 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a8a2b-9738-4cb9-a3d5-26246ae428dc None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:38.334689 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c81ac2-4ff4-4a60-bf01-27ed0d8b077c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:38.335340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c81ac2-4ff4-4a60-bf01-27ed0d8b077c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:38.335629 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c81ac2-4ff4-4a60-bf01-27ed0d8b077c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:38.335908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c81ac2-4ff4-4a60-bf01-27ed0d8b077c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:38.406116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c81ac2-4ff4-4a60-bf01-27ed0d8b077c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:38.407892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c81ac2-4ff4-4a60-bf01-27ed0d8b077c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:38.517566 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 163/316] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:38.523842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4e230-8ed4-43fd-ad98-0478add7f780 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:38.636689 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4e230-8ed4-43fd-ad98-0478add7f780 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:38.670871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27cf927-0711-4e78-8650-db760ed998d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:38.672068 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27cf927-0711-4e78-8650-db760ed998d5 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:38.672545 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27cf927-0711-4e78-8650-db760ed998d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:38.673011 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27cf927-0711-4e78-8650-db760ed998d5 None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:38.674433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27cf927-0711-4e78-8650-db760ed998d5 None admin] RBAC: Authorizing `identity:get_project(project_id=775bb0dd255b49ca81b1452cbe667589)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:38.676063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27cf927-0711-4e78-8650-db760ed998d5 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:38.678932 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 154/317] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:38 2022] GET /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:38.684767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c81ac2-4ff4-4a60-bf01-27ed0d8b077c None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:38.782789 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c81ac2-4ff4-4a60-bf01-27ed0d8b077c None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:38.817098 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd800e-acce-47cd-bc15-9522339012f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:38.818133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd800e-acce-47cd-bc15-9522339012f7 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:38.818613 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd800e-acce-47cd-bc15-9522339012f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:38.819096 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd800e-acce-47cd-bc15-9522339012f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:38.820124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd800e-acce-47cd-bc15-9522339012f7 None admin] RBAC: Adding query filter params (role.id=f7a4054724ca4163960cf635d13b055f, scope.project.id=775bb0dd255b49ca81b1452cbe667589, user.id=6ccfbf763b834334b259636e317cb678) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:38.820641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd800e-acce-47cd-bc15-9522339012f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:38.822275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd800e-acce-47cd-bc15-9522339012f7 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:38.829026 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 164/318] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:57:38 2022] GET /identity/v3/role_assignments?user.id=6ccfbf763b834334b259636e317cb678&scope.project.id=775bb0dd255b49ca81b1452cbe667589&role.id=f7a4054724ca4163960cf635d13b055f => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:39.702365 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9f658-a4f7-44f2-a1fb-9bab8c50b37a None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:39.702900 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9f658-a4f7-44f2-a1fb-9bab8c50b37a None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:39.702900 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9f658-a4f7-44f2-a1fb-9bab8c50b37a None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:39.704410 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 155/319] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:39.709474 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bed576-ebad-474e-8509-eb7a44eec2b7 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:39.709855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bed576-ebad-474e-8509-eb7a44eec2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:39.710031 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bed576-ebad-474e-8509-eb7a44eec2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:39.726250 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-a1bed576-ebad-474e-8509-eb7a44eec2b7 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:39.727422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a1bed576-ebad-474e-8509-eb7a44eec2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:39.792497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 165/320] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:39.815153 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9f658-a4f7-44f2-a1fb-9bab8c50b37a None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:39.914436 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9f658-a4f7-44f2-a1fb-9bab8c50b37a None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:39.959473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3db762-d0d8-4392-adff-27fdf6efc554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:39.959473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3db762-d0d8-4392-adff-27fdf6efc554 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:39.959473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3db762-d0d8-4392-adff-27fdf6efc554 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:39.959778 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3db762-d0d8-4392-adff-27fdf6efc554 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:39.960916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3db762-d0d8-4392-adff-27fdf6efc554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:39.962423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3db762-d0d8-4392-adff-27fdf6efc554 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:39.964266 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 156/321] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:57:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:39.969985 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bed576-ebad-474e-8509-eb7a44eec2b7 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:39.983156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51c195-4f1d-49ed-934f-af34b8e677b3 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:39.983448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51c195-4f1d-49ed-934f-af34b8e677b3 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:39.983729 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51c195-4f1d-49ed-934f-af34b8e677b3 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:39.985221 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 157/322] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:57:39.990403 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2c84f-ca57-4ca4-9862-97195b20c741 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:39.990686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2c84f-ca57-4ca4-9862-97195b20c741 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:39.990958 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2c84f-ca57-4ca4-9862-97195b20c741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:40.007352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-b1a2c84f-ca57-4ca4-9862-97195b20c741 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:40.008572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b1a2c84f-ca57-4ca4-9862-97195b20c741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:40.069662 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bed576-ebad-474e-8509-eb7a44eec2b7 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:40.075816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 158/323] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:57:40.097554 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2c84f-ca57-4ca4-9862-97195b20c741 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:40.103591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31317c-3495-45ac-b41e-46d2a311a22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:40.104424 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31317c-3495-45ac-b41e-46d2a311a22f None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:40.104876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31317c-3495-45ac-b41e-46d2a311a22f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:40.105287 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31317c-3495-45ac-b41e-46d2a311a22f None admin] PATH_INFO: `/v3/groups` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:40.106606 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31317c-3495-45ac-b41e-46d2a311a22f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:40.108376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31317c-3495-45ac-b41e-46d2a311a22f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:40.197256 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2c84f-ca57-4ca4-9862-97195b20c741 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:40.213294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 166/324] 10.222.0.65 () {60 vars in 1134 bytes} [Tue May 10 19:57:39 2022] POST /identity/v3/groups => generated 226 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:57:40.233063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c30e0a-0feb-45f5-8537-0e9b4cfa82c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:40.233890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c30e0a-0feb-45f5-8537-0e9b4cfa82c6 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:40.234204 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c30e0a-0feb-45f5-8537-0e9b4cfa82c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:40.234486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c30e0a-0feb-45f5-8537-0e9b4cfa82c6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:40.246835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c30e0a-0feb-45f5-8537-0e9b4cfa82c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:40.254728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-72c30e0a-0feb-45f5-8537-0e9b4cfa82c6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 19:57:40.254728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:40.254728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:40.254728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:40.254728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:40.254728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:40.254728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:40.254728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:40.254728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:40.254728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:40.254728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:40.254728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:40.254728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:40.254728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:40.254728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:40.254728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:40.255718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:40.255718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:40.255718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:40.255718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:40.255718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:40.255718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:40.255718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:40.255718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:40.255718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:40.255718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:40.255718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:40.255718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:40.255718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:40.255718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:40.255718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:40.255718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:40.261467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:40.261467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:40.261467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:40.261467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:40.261467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:40.261467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:40.261467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:40.261467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:40.261467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:40.261467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:40.261467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 19:57:40.261467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:40.261467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 159/325] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:57:40.263762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31317c-3495-45ac-b41e-46d2a311a22f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:40.337551 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d0496-ca28-4acf-89e8-5bbe4844f8b5 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:40.337857 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d0496-ca28-4acf-89e8-5bbe4844f8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:40.338167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d0496-ca28-4acf-89e8-5bbe4844f8b5 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:40.339550 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 160/326] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:40.345252 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08421c-753a-4651-ba73-71476aa9d5cc None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:40.345538 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08421c-753a-4651-ba73-71476aa9d5cc None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:40.345817 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08421c-753a-4651-ba73-71476aa9d5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:40.363453 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-cb08421c-753a-4651-ba73-71476aa9d5cc None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:40.364743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-cb08421c-753a-4651-ba73-71476aa9d5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:40.423481 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31317c-3495-45ac-b41e-46d2a311a22f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:40.437475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 161/327] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:57:40.458901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f3c07-0550-41da-8707-c8b16bd5a8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:40.459846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f3c07-0550-41da-8707-c8b16bd5a8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:40.460126 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f3c07-0550-41da-8707-c8b16bd5a8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:40.460221 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08421c-753a-4651-ba73-71476aa9d5cc None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:40.460402 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f3c07-0550-41da-8707-c8b16bd5a8c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:40.461160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f3c07-0550-41da-8707-c8b16bd5a8c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:40.461464 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f3c07-0550-41da-8707-c8b16bd5a8c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:40.462866 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f3c07-0550-41da-8707-c8b16bd5a8c7 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:40.471874 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 167/328] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:40.479620 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f3c07-0550-41da-8707-c8b16bd5a8c7 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:40.564182 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08421c-753a-4651-ba73-71476aa9d5cc None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:40.603495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77c437-c56b-4982-b911-0fce53e64aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:40.604308 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77c437-c56b-4982-b911-0fce53e64aaf None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:40.604587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77c437-c56b-4982-b911-0fce53e64aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:40.604858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77c437-c56b-4982-b911-0fce53e64aaf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:40.610943 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f3c07-0550-41da-8707-c8b16bd5a8c7 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:40.616725 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77c437-c56b-4982-b911-0fce53e64aaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:40.624812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-9f77c437-c56b-4982-b911-0fce53e64aaf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:40.624812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:40.624812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:40.624812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:40.624812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:40.624812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:40.624812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:40.624812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:40.624812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:40.624812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:40.624812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:40.624812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:40.624812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:40.624812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:40.624812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:40.624812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:40.625787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:40.625787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:40.625787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:40.625787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:40.625787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:40.625787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:40.625787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:40.625787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:40.625787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:40.625787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:40.625787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:40.625787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:40.625787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:40.625787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:40.625787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:40.625787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:40.627495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:40.627495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:40.627495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:40.627495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:40.627495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:40.627495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:40.627495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:40.627495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:40.627495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:40.627495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:40.627495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:40.627495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:40.627495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 162/329] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:57:40.633227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77c437-c56b-4982-b911-0fce53e64aaf None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:40.657116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6b281-ef27-4e95-8ade-f95078b19aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:40.658003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6b281-ef27-4e95-8ade-f95078b19aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:40.658309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6b281-ef27-4e95-8ade-f95078b19aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:40.658582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6b281-ef27-4e95-8ade-f95078b19aa8 None admin] PATH_INFO: `/v3/users/f2f4ead7f8d24073bd668d70b83e10cc` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:40.660073 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6b281-ef27-4e95-8ade-f95078b19aa8 None admin] RBAC: Authorizing `identity:get_user(user_id=f2f4ead7f8d24073bd668d70b83e10cc)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:40.661561 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6b281-ef27-4e95-8ade-f95078b19aa8 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:40.664307 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 168/330] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:40 2022] GET /identity/v3/users/f2f4ead7f8d24073bd668d70b83e10cc => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:40.669862 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6b281-ef27-4e95-8ade-f95078b19aa8 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:40.737076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77c437-c56b-4982-b911-0fce53e64aaf None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:40.771890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84366e9c-2276-4008-8d58-404028735405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:40.772563 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84366e9c-2276-4008-8d58-404028735405 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:40.772848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84366e9c-2276-4008-8d58-404028735405 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:40.773130 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84366e9c-2276-4008-8d58-404028735405 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:40.773900 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84366e9c-2276-4008-8d58-404028735405 None admin] RBAC: Adding query filter params (name=member) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:40.774240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84366e9c-2276-4008-8d58-404028735405 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:40.775416 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6b281-ef27-4e95-8ade-f95078b19aa8 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:40.775906 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84366e9c-2276-4008-8d58-404028735405 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:40.784372 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 163/331] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:40.789804 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84366e9c-2276-4008-8d58-404028735405 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:40.809940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca5962-505e-443a-8a49-6ae035759afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:40.810986 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca5962-505e-443a-8a49-6ae035759afc None admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:40.811266 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca5962-505e-443a-8a49-6ae035759afc None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:40.811579 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca5962-505e-443a-8a49-6ae035759afc None admin] PATH_INFO: `/v3/system/users/f2f4ead7f8d24073bd668d70b83e10cc/roles/f7a4054724ca4163960cf635d13b055f` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:40.813188 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca5962-505e-443a-8a49-6ae035759afc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f2f4ead7f8d24073bd668d70b83e10cc, role_id=f7a4054724ca4163960cf635d13b055f)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:40.814774 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca5962-505e-443a-8a49-6ae035759afc None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:40.892311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84366e9c-2276-4008-8d58-404028735405 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:40.905365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 169/332] 10.222.0.65 () {58 vars in 1334 bytes} [Tue May 10 19:57:40 2022] PUT /identity/v3/system/users/f2f4ead7f8d24073bd668d70b83e10cc/roles/f7a4054724ca4163960cf635d13b055f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:57:40.931807 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501b3c9-82ff-4481-af5f-2ab10ea7a3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:40.932757 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501b3c9-82ff-4481-af5f-2ab10ea7a3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:40.933086 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501b3c9-82ff-4481-af5f-2ab10ea7a3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:40.933360 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501b3c9-82ff-4481-af5f-2ab10ea7a3c4 None admin] PATH_INFO: `/v3/users/07313b26b7d8455ab4d49b7ea57a1956` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:40.934809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501b3c9-82ff-4481-af5f-2ab10ea7a3c4 None admin] RBAC: Authorizing `identity:get_user(user_id=07313b26b7d8455ab4d49b7ea57a1956)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:40.936414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501b3c9-82ff-4481-af5f-2ab10ea7a3c4 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:40.938492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 164/333] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:40 2022] GET /identity/v3/users/07313b26b7d8455ab4d49b7ea57a1956 => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:40.947545 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca5962-505e-443a-8a49-6ae035759afc None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:41.064497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca5962-505e-443a-8a49-6ae035759afc None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:41.105700 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de593cd3-89ac-49ce-a859-ec07d065ce98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:41.106459 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de593cd3-89ac-49ce-a859-ec07d065ce98 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:41.106735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de593cd3-89ac-49ce-a859-ec07d065ce98 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:41.107060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de593cd3-89ac-49ce-a859-ec07d065ce98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:41.107964 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de593cd3-89ac-49ce-a859-ec07d065ce98 None admin] RBAC: Adding query filter params (role.id=5f62ce25636443f8a5add67f208afe62, user.id=07313b26b7d8455ab4d49b7ea57a1956, scope.system=all) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:41.108276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de593cd3-89ac-49ce-a859-ec07d065ce98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:41.109734 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de593cd3-89ac-49ce-a859-ec07d065ce98 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:41.123334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 170/334] 10.222.0.65 () {56 vars in 1308 bytes} [Tue May 10 19:57:40 2022] GET /identity/v3/role_assignments?user.id=07313b26b7d8455ab4d49b7ea57a1956&scope.system=all&role.id=5f62ce25636443f8a5add67f208afe62 => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:42.810123 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a950ef7-5585-4aeb-90bc-b2343efd4bfb None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:42.810123 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a950ef7-5585-4aeb-90bc-b2343efd4bfb None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:42.810123 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a950ef7-5585-4aeb-90bc-b2343efd4bfb None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:42.811228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 165/335] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:42.816964 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4dd9ee-8fdc-4b03-a7ed-536f9a40c87d None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:42.818040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4dd9ee-8fdc-4b03-a7ed-536f9a40c87d None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:42.818216 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4dd9ee-8fdc-4b03-a7ed-536f9a40c87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:42.843153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-ea4dd9ee-8fdc-4b03-a7ed-536f9a40c87d None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:42.844356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-ea4dd9ee-8fdc-4b03-a7ed-536f9a40c87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:42.932382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 171/336] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:42.972938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a950ef7-5585-4aeb-90bc-b2343efd4bfb None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:43.099550 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a950ef7-5585-4aeb-90bc-b2343efd4bfb None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:43.147184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ed0c7-e456-407b-998d-c5def87d7efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:43.148001 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ed0c7-e456-407b-998d-c5def87d7efc None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:43.148286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ed0c7-e456-407b-998d-c5def87d7efc None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:43.148566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ed0c7-e456-407b-998d-c5def87d7efc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:43.159551 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ed0c7-e456-407b-998d-c5def87d7efc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:43.167904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-922ed0c7-e456-407b-998d-c5def87d7efc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:43.167904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:43.167904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:43.167904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:43.167904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:43.167904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:43.167904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:43.167904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:43.167904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:43.167904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:43.167904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:43.167904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:43.167904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:43.167904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:43.167904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:43.167904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:43.168820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:43.168820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:43.168820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:43.168820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:43.168820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:43.168820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:43.168820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:43.168820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:43.168820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:43.168820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:43.168820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:43.168820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:43.168820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:43.168820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:43.168820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:43.168820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:43.170065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:43.170065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:43.170065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:43.170065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:43.170065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:43.170065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:43.170065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:43.170065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:43.170065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:43.170065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:43.170065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:43.170065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:43.170065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 166/337] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:43.177270 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4dd9ee-8fdc-4b03-a7ed-536f9a40c87d None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:43.208958 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3338a-3c58-4229-8d74-22882eb9403d None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:43.209295 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3338a-3c58-4229-8d74-22882eb9403d None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:43.209526 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3338a-3c58-4229-8d74-22882eb9403d None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:43.211193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 167/338] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:43.216181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5efc8-73d9-41b3-a49c-7602283cac6c None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:43.216469 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5efc8-73d9-41b3-a49c-7602283cac6c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:43.216748 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5efc8-73d9-41b3-a49c-7602283cac6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:43.259141 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-0ca5efc8-73d9-41b3-a49c-7602283cac6c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:43.260362 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0ca5efc8-73d9-41b3-a49c-7602283cac6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:43.318617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4dd9ee-8fdc-4b03-a7ed-536f9a40c87d None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:43.369510 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d286d7-2d9a-4464-ba4a-db63641c9c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:43.370192 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d286d7-2d9a-4464-ba4a-db63641c9c71 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:43.370478 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d286d7-2d9a-4464-ba4a-db63641c9c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:43.370750 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d286d7-2d9a-4464-ba4a-db63641c9c71 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:43.371511 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d286d7-2d9a-4464-ba4a-db63641c9c71 None admin] RBAC: Adding query filter params (name=member) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:43.371812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d286d7-2d9a-4464-ba4a-db63641c9c71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:43.373189 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d286d7-2d9a-4464-ba4a-db63641c9c71 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:43.385744 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 172/339] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:43.393062 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d286d7-2d9a-4464-ba4a-db63641c9c71 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:43.447755 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 168/340] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:57:43.478677 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5efc8-73d9-41b3-a49c-7602283cac6c None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:43.531010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d286d7-2d9a-4464-ba4a-db63641c9c71 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:43.573074 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba3457-8add-4179-81db-4627d228a5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:43.574043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba3457-8add-4179-81db-4627d228a5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:43.574528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba3457-8add-4179-81db-4627d228a5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:43.574978 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba3457-8add-4179-81db-4627d228a5c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:43.692835 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba3457-8add-4179-81db-4627d228a5c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:43.694335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba3457-8add-4179-81db-4627d228a5c2 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:43.733756 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5efc8-73d9-41b3-a49c-7602283cac6c None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:43.812194 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b99df-325f-45df-b21b-6296dc923325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:43.813044 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b99df-325f-45df-b21b-6296dc923325 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:43.813301 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b99df-325f-45df-b21b-6296dc923325 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:43.813577 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b99df-325f-45df-b21b-6296dc923325 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:43.831199 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b99df-325f-45df-b21b-6296dc923325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:43.844005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-5a3b99df-325f-45df-b21b-6296dc923325 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:43.844005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:43.844005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:43.844005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:43.844005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:43.844005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:43.844005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:43.844005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:43.844005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:43.844005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:43.844005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:43.844005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:43.844005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:43.844005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:43.844005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:43.844005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:43.844918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:43.844918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:43.844918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:43.844918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:43.844918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:43.844918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:43.844918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:43.844918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:43.844918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:43.844918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:43.844918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:43.844918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:43.844918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:43.844918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:43.844918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:43.844918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:43.845705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:43.845705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:43.845705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:43.845705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:43.845705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:43.845705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:43.845705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:43.845705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:43.845705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:43.845705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:43.845705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:43.845705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:43.845705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 169/341] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:57:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:57:43.855879 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b99df-325f-45df-b21b-6296dc923325 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:43.873779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 173/342] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 19:57:43.875181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b96cb-eb30-4ec2-98a9-39b3214c2f85 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:43.875467 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b96cb-eb30-4ec2-98a9-39b3214c2f85 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:43.875743 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b96cb-eb30-4ec2-98a9-39b3214c2f85 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:43.877005 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 174/343] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:43.882975 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b96cb-eb30-4ec2-98a9-39b3214c2f85 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:43.975412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b99df-325f-45df-b21b-6296dc923325 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:44.015120 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a533ae-8877-4073-8caf-2c2bea3a00e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:44.016210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a533ae-8877-4073-8caf-2c2bea3a00e3 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:44.016709 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a533ae-8877-4073-8caf-2c2bea3a00e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:44.017945 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a533ae-8877-4073-8caf-2c2bea3a00e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:44.018928 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a533ae-8877-4073-8caf-2c2bea3a00e3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:44.019597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a533ae-8877-4073-8caf-2c2bea3a00e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:44.022885 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a533ae-8877-4073-8caf-2c2bea3a00e3 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:44.041054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 170/344] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:57:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:44.046110 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a760a7b-2165-4368-8ead-4a56362eb893 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:44.046110 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a760a7b-2165-4368-8ead-4a56362eb893 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:44.046110 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a760a7b-2165-4368-8ead-4a56362eb893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:44.061279 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b96cb-eb30-4ec2-98a9-39b3214c2f85 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:44.071854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-6a760a7b-2165-4368-8ead-4a56362eb893 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:44.073047 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-6a760a7b-2165-4368-8ead-4a56362eb893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:44.123326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22298878-db69-4a51-9f70-eb3bdbb10937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:44.124559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22298878-db69-4a51-9f70-eb3bdbb10937 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:44.125050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22298878-db69-4a51-9f70-eb3bdbb10937 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:44.125530 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22298878-db69-4a51-9f70-eb3bdbb10937 None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:44.127076 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22298878-db69-4a51-9f70-eb3bdbb10937 None admin] RBAC: Authorizing `identity:get_project(project_id=675b1b490743424e9d44dd2e484dfa95)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:44.129869 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22298878-db69-4a51-9f70-eb3bdbb10937 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:44.133408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 175/345] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:43 2022] GET /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:44.135663 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22298878-db69-4a51-9f70-eb3bdbb10937 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:44.194134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 171/346] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:44.194989 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a760a7b-2165-4368-8ead-4a56362eb893 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:44.251403 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22298878-db69-4a51-9f70-eb3bdbb10937 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:44.312953 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2e329-95b0-440b-9d22-f0c74f390302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:44.313897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2e329-95b0-440b-9d22-f0c74f390302 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:44.314407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2e329-95b0-440b-9d22-f0c74f390302 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:44.314817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2e329-95b0-440b-9d22-f0c74f390302 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:44.372332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a760a7b-2165-4368-8ead-4a56362eb893 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:44.397833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2e329-95b0-440b-9d22-f0c74f390302 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:44.399594 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2e329-95b0-440b-9d22-f0c74f390302 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:44.453917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41363b-15a9-4de3-9246-3e8ae5e84892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:44.454820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41363b-15a9-4de3-9246-3e8ae5e84892 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:44.455101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41363b-15a9-4de3-9246-3e8ae5e84892 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:44.455376 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41363b-15a9-4de3-9246-3e8ae5e84892 None admin] PATH_INFO: `/v3/groups/b119fac6c6f04c74baf3a690c852feea` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:44.471593 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41363b-15a9-4de3-9246-3e8ae5e84892 None admin] RBAC: Authorizing `identity:get_group(group_id=b119fac6c6f04c74baf3a690c852feea)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:44.473061 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41363b-15a9-4de3-9246-3e8ae5e84892 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:44.478321 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 172/347] 10.222.0.65 () {56 vars in 1180 bytes} [Tue May 10 19:57:44 2022] GET /identity/v3/groups/b119fac6c6f04c74baf3a690c852feea => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:44.479208 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41363b-15a9-4de3-9246-3e8ae5e84892 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:44.503053 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 176/348] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:44.503820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2e329-95b0-440b-9d22-f0c74f390302 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:44.608776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2e329-95b0-440b-9d22-f0c74f390302 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:44.643482 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a7398-0d3a-4545-bf14-c9b1bb563711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:44.644137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a7398-0d3a-4545-bf14-c9b1bb563711 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:44.644415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a7398-0d3a-4545-bf14-c9b1bb563711 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:44.644690 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a7398-0d3a-4545-bf14-c9b1bb563711 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:44.645562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a7398-0d3a-4545-bf14-c9b1bb563711 None admin] RBAC: Adding query filter params (group.id=b119fac6c6f04c74baf3a690c852feea, role.id=5f62ce25636443f8a5add67f208afe62, scope.project.id=675b1b490743424e9d44dd2e484dfa95) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:44.645869 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a7398-0d3a-4545-bf14-c9b1bb563711 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:44.647331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a7398-0d3a-4545-bf14-c9b1bb563711 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:44.653385 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 177/349] 10.222.0.65 () {56 vars in 1376 bytes} [Tue May 10 19:57:44 2022] GET /identity/v3/role_assignments?group.id=b119fac6c6f04c74baf3a690c852feea&scope.project.id=675b1b490743424e9d44dd2e484dfa95&role.id=5f62ce25636443f8a5add67f208afe62 => generated 260 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:44.654362 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a7398-0d3a-4545-bf14-c9b1bb563711 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:44.728317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41363b-15a9-4de3-9246-3e8ae5e84892 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:44.771546 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a7398-0d3a-4545-bf14-c9b1bb563711 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:44.813339 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2dc47-3418-416d-9406-3cbf963d71d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:44.814296 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2dc47-3418-416d-9406-3cbf963d71d5 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:44.814575 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2dc47-3418-416d-9406-3cbf963d71d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:44.814846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2dc47-3418-416d-9406-3cbf963d71d5 None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:44.816089 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2dc47-3418-416d-9406-3cbf963d71d5 None admin] RBAC: Authorizing `identity:get_project(project_id=675b1b490743424e9d44dd2e484dfa95)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:44.817564 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2dc47-3418-416d-9406-3cbf963d71d5 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:44.819474 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 178/350] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:44 2022] GET /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:44.824886 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2dc47-3418-416d-9406-3cbf963d71d5 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:44.831824 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c8a2f-c374-4706-a310-a325a6a49d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:44.833166 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c8a2f-c374-4706-a310-a325a6a49d22 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:44.833465 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c8a2f-c374-4706-a310-a325a6a49d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:44.833741 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c8a2f-c374-4706-a310-a325a6a49d22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:44.855207 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c8a2f-c374-4706-a310-a325a6a49d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:44.871275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-9a6c8a2f-c374-4706-a310-a325a6a49d22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:44.871275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:44.871275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:44.871275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:44.871275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:44.871275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:44.871275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:44.871275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:44.871275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:44.871275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:44.871275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:44.871275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:44.871275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:44.871275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:44.871275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:44.871275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:44.873494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:44.873494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:44.873494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:44.873494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:44.873494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:44.873494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:44.873494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:44.873494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:44.873494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:44.873494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:44.873494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:44.873494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:44.873494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:44.873494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:44.873494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:44.873494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:44.875516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:44.875516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:44.875516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:44.875516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:44.875516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:44.875516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:44.875516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:44.875516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:44.875516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:44.875516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:44.875516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:44.875516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:44.875516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 173/351] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:44.883161 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c8a2f-c374-4706-a310-a325a6a49d22 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:44.937064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2dc47-3418-416d-9406-3cbf963d71d5 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:44.976935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6672c-473c-40ec-9119-788776ec74e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:44.977800 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6672c-473c-40ec-9119-788776ec74e1 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:44.978097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6672c-473c-40ec-9119-788776ec74e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:44.978381 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6672c-473c-40ec-9119-788776ec74e1 None admin] PATH_INFO: `/v3/groups/b119fac6c6f04c74baf3a690c852feea` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:44.979613 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6672c-473c-40ec-9119-788776ec74e1 None admin] RBAC: Authorizing `identity:get_group(group_id=b119fac6c6f04c74baf3a690c852feea)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:44.981060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6672c-473c-40ec-9119-788776ec74e1 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:44.983154 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 179/352] 10.222.0.65 () {56 vars in 1180 bytes} [Tue May 10 19:57:44 2022] GET /identity/v3/groups/b119fac6c6f04c74baf3a690c852feea => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:44.989110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6672c-473c-40ec-9119-788776ec74e1 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:45.096229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6672c-473c-40ec-9119-788776ec74e1 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:45.131900 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fedc43-632b-414f-990c-020ae1954860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:45.132474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c8a2f-c374-4706-a310-a325a6a49d22 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:45.132592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fedc43-632b-414f-990c-020ae1954860 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:45.132874 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fedc43-632b-414f-990c-020ae1954860 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:45.133144 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fedc43-632b-414f-990c-020ae1954860 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:45.134034 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fedc43-632b-414f-990c-020ae1954860 None admin] RBAC: Adding query filter params (group.id=b119fac6c6f04c74baf3a690c852feea, role.id=4fbb0a346f5f4f36bb32664ee799db42, scope.project.id=675b1b490743424e9d44dd2e484dfa95) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:45.134372 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fedc43-632b-414f-990c-020ae1954860 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:45.135798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fedc43-632b-414f-990c-020ae1954860 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:45.144771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 180/353] 10.222.0.65 () {56 vars in 1376 bytes} [Tue May 10 19:57:44 2022] GET /identity/v3/role_assignments?group.id=b119fac6c6f04c74baf3a690c852feea&scope.project.id=675b1b490743424e9d44dd2e484dfa95&role.id=4fbb0a346f5f4f36bb32664ee799db42 => generated 260 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:45.201089 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d530fc-92b9-4f9b-8c4e-e937fed9d17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:45.202217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d530fc-92b9-4f9b-8c4e-e937fed9d17e None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:45.202713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d530fc-92b9-4f9b-8c4e-e937fed9d17e None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:45.203197 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d530fc-92b9-4f9b-8c4e-e937fed9d17e None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:45.204215 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d530fc-92b9-4f9b-8c4e-e937fed9d17e None admin] RBAC: Adding query filter params (name=member) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:45.204748 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d530fc-92b9-4f9b-8c4e-e937fed9d17e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:45.206415 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d530fc-92b9-4f9b-8c4e-e937fed9d17e None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:45.206794 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc940f9b-43a4-4d9d-8a37-503abcde0329 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:45.207085 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc940f9b-43a4-4d9d-8a37-503abcde0329 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:45.207365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc940f9b-43a4-4d9d-8a37-503abcde0329 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:45.214126 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 181/354] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:45.215359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f85cd-dcf2-4298-b493-e2ebbbea87e8 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:45.217136 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f85cd-dcf2-4298-b493-e2ebbbea87e8 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:45.217418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f85cd-dcf2-4298-b493-e2ebbbea87e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:45.223398 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 174/355] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:45.229067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d530fc-92b9-4f9b-8c4e-e937fed9d17e None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:45.241273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-ab2f85cd-dcf2-4298-b493-e2ebbbea87e8 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:45.243902 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-ab2f85cd-dcf2-4298-b493-e2ebbbea87e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:45.323040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 182/356] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:45.324472 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18182de0-bcf6-420b-8550-f4a81185881e None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:45.324749 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18182de0-bcf6-420b-8550-f4a81185881e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:45.325025 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18182de0-bcf6-420b-8550-f4a81185881e None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:45.326688 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 183/357] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:45.339772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e41fc-c60a-4be7-9be1-482e10f88b1c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:45.340090 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e41fc-c60a-4be7-9be1-482e10f88b1c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:45.340334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e41fc-c60a-4be7-9be1-482e10f88b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:45.364883 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-856e41fc-c60a-4be7-9be1-482e10f88b1c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:45.366000 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-856e41fc-c60a-4be7-9be1-482e10f88b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:45.420871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d530fc-92b9-4f9b-8c4e-e937fed9d17e None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:45.453780 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 184/358] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:57:45.454767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e41fc-c60a-4be7-9be1-482e10f88b1c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:45.469937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f7b13-663b-4530-83e0-629b6562f64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:45.471086 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f7b13-663b-4530-83e0-629b6562f64f None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:45.471574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f7b13-663b-4530-83e0-629b6562f64f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:45.472050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f7b13-663b-4530-83e0-629b6562f64f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:45.556236 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f7b13-663b-4530-83e0-629b6562f64f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:45.558406 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f7b13-663b-4530-83e0-629b6562f64f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:45.578822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e41fc-c60a-4be7-9be1-482e10f88b1c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:45.622509 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401690e3-974b-4130-b4ec-ffb8e5691d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:45.623239 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401690e3-974b-4130-b4ec-ffb8e5691d59 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:45.623548 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401690e3-974b-4130-b4ec-ffb8e5691d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:45.623949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401690e3-974b-4130-b4ec-ffb8e5691d59 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:45.637686 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401690e3-974b-4130-b4ec-ffb8e5691d59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:45.647075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-401690e3-974b-4130-b4ec-ffb8e5691d59 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 19:57:45.647075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:45.647075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:45.647075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:45.647075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:45.647075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:45.647075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:45.647075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:45.647075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:45.647075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:45.647075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:45.647075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:45.647075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:45.647075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:45.647075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:45.647075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:45.648008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:45.648008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:45.648008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:45.648008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:45.648008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:45.648008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:45.648008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:45.648008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:45.648008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:45.648008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:45.648008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:45.648008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:45.648008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:45.648008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:45.648008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:45.648008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:45.648803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:45.648803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:45.648803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:45.648803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:45.648803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:45.648803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:45.648803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:45.648803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:45.648803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:45.648803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:45.648803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 19:57:45.648803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:45.648803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 185/359] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:45.650251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b36d27-548c-429c-a27b-ec0d3f5cc39e None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:45.650251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b36d27-548c-429c-a27b-ec0d3f5cc39e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:45.650251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b36d27-548c-429c-a27b-ec0d3f5cc39e None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:45.655716 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 186/360] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:45.656756 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b36d27-548c-429c-a27b-ec0d3f5cc39e None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:45.681241 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 175/361] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 19:57:45.682098 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f7b13-663b-4530-83e0-629b6562f64f None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:45.781600 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b36d27-548c-429c-a27b-ec0d3f5cc39e None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:45.798914 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f7b13-663b-4530-83e0-629b6562f64f None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:45.822370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21803fb0-9b93-4a98-b67c-7c9900d2bee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:45.823150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21803fb0-9b93-4a98-b67c-7c9900d2bee1 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:45.823473 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21803fb0-9b93-4a98-b67c-7c9900d2bee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:45.823749 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21803fb0-9b93-4a98-b67c-7c9900d2bee1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:45.835326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21803fb0-9b93-4a98-b67c-7c9900d2bee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:45.842131 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad10ad2-9642-462d-9df7-1a17e75812cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:45.843404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad10ad2-9642-462d-9df7-1a17e75812cd None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:45.843833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-21803fb0-9b93-4a98-b67c-7c9900d2bee1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:45.843833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:45.843833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:45.843833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:45.843833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:45.843833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:45.843833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:45.843833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:45.843833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:45.843833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:45.843833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:45.843833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:45.843833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:45.843833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:45.843833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:45.843833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:45.844697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:45.844697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:45.844697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:45.844697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:45.844697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:45.844697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:45.844697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:45.844697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:45.844697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:45.844697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:45.844697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:45.844697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:45.844697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:45.844697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:45.844697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:45.844697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:45.845929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:45.845929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:45.845929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:45.845929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:45.845929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:45.845929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:45.845929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:45.845929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:45.845929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:45.845929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:45.845929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:45.845929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:45.845929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 187/362] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:57:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:45.846978 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdc41c-3070-4eea-84d3-ed3a19d70ea7 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:45.847219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad10ad2-9642-462d-9df7-1a17e75812cd None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:45.847314 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdc41c-3070-4eea-84d3-ed3a19d70ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:45.847561 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdc41c-3070-4eea-84d3-ed3a19d70ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:45.847983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad10ad2-9642-462d-9df7-1a17e75812cd None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:45.849498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad10ad2-9642-462d-9df7-1a17e75812cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:45.850004 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad10ad2-9642-462d-9df7-1a17e75812cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:45.851992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad10ad2-9642-462d-9df7-1a17e75812cd None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:45.866271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 176/363] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:45.867336 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad10ad2-9642-462d-9df7-1a17e75812cd None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:45.868654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-dccdc41c-3070-4eea-84d3-ed3a19d70ea7 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:45.871764 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-dccdc41c-3070-4eea-84d3-ed3a19d70ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:45.942287 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 188/364] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:45.943164 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdc41c-3070-4eea-84d3-ed3a19d70ea7 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:45.998889 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad10ad2-9642-462d-9df7-1a17e75812cd None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:46.041372 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a36bdd-e715-48f6-b4d9-b387bda42540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:46.043192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a36bdd-e715-48f6-b4d9-b387bda42540 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:46.043693 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a36bdd-e715-48f6-b4d9-b387bda42540 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:46.044332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a36bdd-e715-48f6-b4d9-b387bda42540 None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:46.046230 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a36bdd-e715-48f6-b4d9-b387bda42540 None admin] RBAC: Authorizing `identity:get_project(project_id=775bb0dd255b49ca81b1452cbe667589)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:46.050210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a36bdd-e715-48f6-b4d9-b387bda42540 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:46.050210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 177/365] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:45 2022] GET /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:46.052535 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a36bdd-e715-48f6-b4d9-b387bda42540 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:46.069390 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdc41c-3070-4eea-84d3-ed3a19d70ea7 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:46.108643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d206d2-ce63-42b3-8f53-603fb2b94aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:46.109314 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d206d2-ce63-42b3-8f53-603fb2b94aab None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:46.109591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d206d2-ce63-42b3-8f53-603fb2b94aab None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:46.109861 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d206d2-ce63-42b3-8f53-603fb2b94aab None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:46.110756 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d206d2-ce63-42b3-8f53-603fb2b94aab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:46.111066 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d206d2-ce63-42b3-8f53-603fb2b94aab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:46.112455 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d206d2-ce63-42b3-8f53-603fb2b94aab None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:46.125605 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 189/366] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:57:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:46.127044 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d206d2-ce63-42b3-8f53-603fb2b94aab None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:46.184097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a36bdd-e715-48f6-b4d9-b387bda42540 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:46.231466 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03684e6c-df28-41cb-99a1-cf78092c6608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:46.232285 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03684e6c-df28-41cb-99a1-cf78092c6608 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:46.232568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03684e6c-df28-41cb-99a1-cf78092c6608 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:46.232848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03684e6c-df28-41cb-99a1-cf78092c6608 None admin] PATH_INFO: `/v3/users/f2f4ead7f8d24073bd668d70b83e10cc` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:46.234183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03684e6c-df28-41cb-99a1-cf78092c6608 None admin] RBAC: Authorizing `identity:get_user(user_id=f2f4ead7f8d24073bd668d70b83e10cc)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:46.235637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03684e6c-df28-41cb-99a1-cf78092c6608 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:46.237429 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 178/367] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:57:46 2022] GET /identity/v3/users/f2f4ead7f8d24073bd668d70b83e10cc => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:46.238029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03684e6c-df28-41cb-99a1-cf78092c6608 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:46.251406 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d206d2-ce63-42b3-8f53-603fb2b94aab None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:46.291009 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ef1a5-080c-419a-9d79-af395100dbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:46.291807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ef1a5-080c-419a-9d79-af395100dbbe None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:46.292095 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ef1a5-080c-419a-9d79-af395100dbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:46.296434 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ef1a5-080c-419a-9d79-af395100dbbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:46.306869 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ef1a5-080c-419a-9d79-af395100dbbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:46.314675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-c87ef1a5-080c-419a-9d79-af395100dbbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:46.314675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:46.314675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:46.314675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:46.314675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:46.314675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:46.314675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:46.314675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:46.314675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:46.314675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:46.314675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:46.314675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:46.314675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:46.314675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:46.314675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:46.314675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:46.315562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:46.315562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:46.315562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:46.315562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:46.315562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:46.315562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:46.315562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:46.315562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:46.315562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:46.315562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:46.315562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:46.315562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:46.315562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:46.315562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:46.315562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:46.315562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:46.316404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:46.316404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:46.316404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:46.316404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:46.316404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:46.316404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:46.316404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:46.316404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:46.316404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:46.316404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:46.316404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:46.316404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:46.316404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 190/368] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:57:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:46.317110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ef1a5-080c-419a-9d79-af395100dbbe None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:46.361265 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03684e6c-df28-41cb-99a1-cf78092c6608 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:46.408002 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfd875-9373-4a7c-8571-f7b151c27a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:46.409158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfd875-9373-4a7c-8571-f7b151c27a50 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:46.409451 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfd875-9373-4a7c-8571-f7b151c27a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:46.409731 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfd875-9373-4a7c-8571-f7b151c27a50 None admin] PATH_INFO: `/v3/groups/b119fac6c6f04c74baf3a690c852feea` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:46.411769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabfd875-9373-4a7c-8571-f7b151c27a50 None admin] RBAC: Authorizing `identity:get_group(group_id=b119fac6c6f04c74baf3a690c852feea)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:46.414064 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabfd875-9373-4a7c-8571-f7b151c27a50 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:46.415731 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 179/369] 10.222.0.65 () {56 vars in 1180 bytes} [Tue May 10 19:57:46 2022] GET /identity/v3/groups/b119fac6c6f04c74baf3a690c852feea => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:46.416503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfd875-9373-4a7c-8571-f7b151c27a50 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:46.446839 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ef1a5-080c-419a-9d79-af395100dbbe None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:46.485370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d8b9d-674a-473c-99db-024a55abe7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:46.485985 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d8b9d-674a-473c-99db-024a55abe7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:46.487314 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d8b9d-674a-473c-99db-024a55abe7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:46.487314 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d8b9d-674a-473c-99db-024a55abe7d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:46.549533 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfd875-9373-4a7c-8571-f7b151c27a50 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:46.564358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d8b9d-674a-473c-99db-024a55abe7d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:46.565889 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d8b9d-674a-473c-99db-024a55abe7d4 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:46.592699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8024340-9e18-409a-8996-d0e8aba1df10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:46.593830 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8024340-9e18-409a-8996-d0e8aba1df10 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:46.594473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8024340-9e18-409a-8996-d0e8aba1df10 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:46.595243 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8024340-9e18-409a-8996-d0e8aba1df10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:46.596548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8024340-9e18-409a-8996-d0e8aba1df10 None admin] RBAC: Adding query filter params (role.id=f7a4054724ca4163960cf635d13b055f, user.id=f2f4ead7f8d24073bd668d70b83e10cc, scope.system=all) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:46.597063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8024340-9e18-409a-8996-d0e8aba1df10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:46.600003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8024340-9e18-409a-8996-d0e8aba1df10 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:46.616904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 180/370] 10.222.0.65 () {56 vars in 1308 bytes} [Tue May 10 19:57:46 2022] GET /identity/v3/role_assignments?user.id=f2f4ead7f8d24073bd668d70b83e10cc&scope.system=all&role.id=f7a4054724ca4163960cf635d13b055f => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:46.617766 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8024340-9e18-409a-8996-d0e8aba1df10 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:46.697138 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 191/371] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:46.698369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d8b9d-674a-473c-99db-024a55abe7d4 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:46.772073 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8024340-9e18-409a-8996-d0e8aba1df10 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:46.821558 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21116435-adb8-4079-a71f-ef456f3f671d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:46.823372 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21116435-adb8-4079-a71f-ef456f3f671d None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:46.824576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21116435-adb8-4079-a71f-ef456f3f671d None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:46.825072 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21116435-adb8-4079-a71f-ef456f3f671d None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:46.826266 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21116435-adb8-4079-a71f-ef456f3f671d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:46.828032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21116435-adb8-4079-a71f-ef456f3f671d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:46.830657 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21116435-adb8-4079-a71f-ef456f3f671d None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:46.843083 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d8b9d-674a-473c-99db-024a55abe7d4 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:46.851537 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 181/372] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:46.853115 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21116435-adb8-4079-a71f-ef456f3f671d None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:46.917897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbf14a-fc7a-4809-811b-f42fad20731c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:46.918999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbf14a-fc7a-4809-811b-f42fad20731c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:46.919507 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbf14a-fc7a-4809-811b-f42fad20731c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:46.920083 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbf14a-fc7a-4809-811b-f42fad20731c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:46.921153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbf14a-fc7a-4809-811b-f42fad20731c None admin] RBAC: Adding query filter params (group.id=b119fac6c6f04c74baf3a690c852feea, role.id=5f62ce25636443f8a5add67f208afe62, scope.project.id=775bb0dd255b49ca81b1452cbe667589) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:46.923496 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbf14a-fc7a-4809-811b-f42fad20731c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:46.925220 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbf14a-fc7a-4809-811b-f42fad20731c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:46.937312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 192/373] 10.222.0.65 () {56 vars in 1376 bytes} [Tue May 10 19:57:46 2022] GET /identity/v3/role_assignments?group.id=b119fac6c6f04c74baf3a690c852feea&scope.project.id=775bb0dd255b49ca81b1452cbe667589&role.id=5f62ce25636443f8a5add67f208afe62 => generated 260 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:46.937989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbf14a-fc7a-4809-811b-f42fad20731c None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:47.033595 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21116435-adb8-4079-a71f-ef456f3f671d None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:47.078856 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b59a2-886e-4bd4-9961-d79e9710629d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:47.080406 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b59a2-886e-4bd4-9961-d79e9710629d None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:47.081153 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b59a2-886e-4bd4-9961-d79e9710629d None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:47.081640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b59a2-886e-4bd4-9961-d79e9710629d None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:47.083705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b59a2-886e-4bd4-9961-d79e9710629d None admin] RBAC: Authorizing `identity:get_project(project_id=775bb0dd255b49ca81b1452cbe667589)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:47.085989 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b59a2-886e-4bd4-9961-d79e9710629d None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:47.088466 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 182/374] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:46 2022] GET /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:47.094788 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b59a2-886e-4bd4-9961-d79e9710629d None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:47.200333 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbf14a-fc7a-4809-811b-f42fad20731c None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:47.219280 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b59a2-886e-4bd4-9961-d79e9710629d None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:47.265663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dabb68-0a3c-4141-a096-367fae739cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:47.266800 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dabb68-0a3c-4141-a096-367fae739cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:47.267256 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dabb68-0a3c-4141-a096-367fae739cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:47.267673 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dabb68-0a3c-4141-a096-367fae739cf4 None admin] PATH_INFO: `/v3/groups/b119fac6c6f04c74baf3a690c852feea` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:47.269070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dabb68-0a3c-4141-a096-367fae739cf4 None admin] RBAC: Authorizing `identity:get_group(group_id=b119fac6c6f04c74baf3a690c852feea)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:47.271282 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dabb68-0a3c-4141-a096-367fae739cf4 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:47.273534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 183/375] 10.222.0.65 () {56 vars in 1180 bytes} [Tue May 10 19:57:47 2022] GET /identity/v3/groups/b119fac6c6f04c74baf3a690c852feea => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:47.279361 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3299a-9f24-477b-ada2-77b2505db55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:47.280041 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3299a-9f24-477b-ada2-77b2505db55e None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:47.280326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3299a-9f24-477b-ada2-77b2505db55e None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:47.280603 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3299a-9f24-477b-ada2-77b2505db55e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:47.285565 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dabb68-0a3c-4141-a096-367fae739cf4 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:47.358891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3299a-9f24-477b-ada2-77b2505db55e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:47.360971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3299a-9f24-477b-ada2-77b2505db55e None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:47.391540 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dabb68-0a3c-4141-a096-367fae739cf4 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:47.426415 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70208e-d57c-45a9-a651-8d367ea4adbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:47.427337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70208e-d57c-45a9-a651-8d367ea4adbf None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:47.427767 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70208e-d57c-45a9-a651-8d367ea4adbf None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:47.428196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70208e-d57c-45a9-a651-8d367ea4adbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:47.429181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee70208e-d57c-45a9-a651-8d367ea4adbf None admin] RBAC: Adding query filter params (group.id=b119fac6c6f04c74baf3a690c852feea, role.id=4fbb0a346f5f4f36bb32664ee799db42, scope.project.id=775bb0dd255b49ca81b1452cbe667589) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:47.430243 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee70208e-d57c-45a9-a651-8d367ea4adbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:47.431849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee70208e-d57c-45a9-a651-8d367ea4adbf None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:47.438144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 184/376] 10.222.0.65 () {56 vars in 1376 bytes} [Tue May 10 19:57:47 2022] GET /identity/v3/role_assignments?group.id=b119fac6c6f04c74baf3a690c852feea&scope.project.id=775bb0dd255b49ca81b1452cbe667589&role.id=4fbb0a346f5f4f36bb32664ee799db42 => generated 260 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:47.505343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 193/377] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:47.507437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5e435-02bc-4973-9ad4-3f8614679819 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:47.507918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5e435-02bc-4973-9ad4-3f8614679819 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:47.508423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5e435-02bc-4973-9ad4-3f8614679819 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:47.509991 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 185/378] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:47.516074 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050d2b3-94ea-47a4-b4dd-5960b8a39c04 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:47.516598 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050d2b3-94ea-47a4-b4dd-5960b8a39c04 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:47.517028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050d2b3-94ea-47a4-b4dd-5960b8a39c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:47.524310 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5e435-02bc-4973-9ad4-3f8614679819 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:47.539826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-0050d2b3-94ea-47a4-b4dd-5960b8a39c04 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:47.541734 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-0050d2b3-94ea-47a4-b4dd-5960b8a39c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:47.677316 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 194/379] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:47.699850 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5e435-02bc-4973-9ad4-3f8614679819 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:47.700914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050d2b3-94ea-47a4-b4dd-5960b8a39c04 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:47.744962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c02344-8927-4d38-a6bf-59bd5f211c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:47.745845 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c02344-8927-4d38-a6bf-59bd5f211c57 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:47.746151 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c02344-8927-4d38-a6bf-59bd5f211c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:47.746431 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c02344-8927-4d38-a6bf-59bd5f211c57 None admin] PATH_INFO: `/v3/projects/475f061ace89472aa69923909a044007` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:47.748233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c02344-8927-4d38-a6bf-59bd5f211c57 None admin] RBAC: Authorizing `identity:get_project(project_id=475f061ace89472aa69923909a044007)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:47.752733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c02344-8927-4d38-a6bf-59bd5f211c57 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:47.752733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 186/380] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:47 2022] GET /identity/v3/projects/475f061ace89472aa69923909a044007 => generated 344 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:47.758145 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c02344-8927-4d38-a6bf-59bd5f211c57 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:47.898348 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050d2b3-94ea-47a4-b4dd-5960b8a39c04 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:47.934429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1989637-ef2d-4e46-8acc-b074f13e1004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:47.935327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1989637-ef2d-4e46-8acc-b074f13e1004 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:47.935610 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1989637-ef2d-4e46-8acc-b074f13e1004 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:47.935880 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1989637-ef2d-4e46-8acc-b074f13e1004 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:47.948928 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1989637-ef2d-4e46-8acc-b074f13e1004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:47.956913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-e1989637-ef2d-4e46-8acc-b074f13e1004 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:47.956913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:47.956913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:47.956913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:47.956913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:47.956913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:47.956913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:47.956913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:47.956913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:47.956913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:47.956913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:47.956913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:47.956913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:47.956913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:47.956913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:47.956913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:47.958399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:47.958399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:47.958399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:47.958399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:47.958399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:47.958399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:47.958399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:47.958399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:47.958399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:47.958399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:47.958399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:47.958399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:47.958399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:47.958399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:47.958399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:47.958399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:47.959217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:47.959217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:47.959217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:47.959217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:47.959217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:47.959217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:47.959217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:47.959217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:47.959217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:47.959217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:47.959217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:47.959217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:47.959217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 195/381] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:57:47.964669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1989637-ef2d-4e46-8acc-b074f13e1004 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:47.966968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c02344-8927-4d38-a6bf-59bd5f211c57 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:48.015206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d1427-203c-4dc7-a465-642690c2d825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:48.016071 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d1427-203c-4dc7-a465-642690c2d825 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:48.016358 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d1427-203c-4dc7-a465-642690c2d825 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:48.016634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d1427-203c-4dc7-a465-642690c2d825 None admin] PATH_INFO: `/v3/groups/122d46298c4d4c6dab814b440bb0eb84` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:48.022307 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d1427-203c-4dc7-a465-642690c2d825 None admin] RBAC: Authorizing `identity:get_group(group_id=122d46298c4d4c6dab814b440bb0eb84)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:48.023785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d1427-203c-4dc7-a465-642690c2d825 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:48.025798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 187/382] 10.222.0.65 () {56 vars in 1180 bytes} [Tue May 10 19:57:47 2022] GET /identity/v3/groups/122d46298c4d4c6dab814b440bb0eb84 => generated 229 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:48.027094 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573ed58-3720-473b-8d10-15a9b8b95c34 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:48.027382 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573ed58-3720-473b-8d10-15a9b8b95c34 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:48.027660 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573ed58-3720-473b-8d10-15a9b8b95c34 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:48.033671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 188/383] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:48.034516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573ed58-3720-473b-8d10-15a9b8b95c34 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:48.072661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1989637-ef2d-4e46-8acc-b074f13e1004 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:48.108481 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50f82c-043b-44a4-9c60-258683a81f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:48.109148 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50f82c-043b-44a4-9c60-258683a81f4f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:48.109486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50f82c-043b-44a4-9c60-258683a81f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:48.109760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50f82c-043b-44a4-9c60-258683a81f4f None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:48.110596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50f82c-043b-44a4-9c60-258683a81f4f None admin] RBAC: Adding query filter params (name=member) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:48.110901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50f82c-043b-44a4-9c60-258683a81f4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:48.112407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50f82c-043b-44a4-9c60-258683a81f4f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:48.121715 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 196/384] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:48.123213 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081361f6-bceb-4fc6-9eba-1f4256f695cc None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:48.123499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081361f6-bceb-4fc6-9eba-1f4256f695cc None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:48.123771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081361f6-bceb-4fc6-9eba-1f4256f695cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:48.143186 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-081361f6-bceb-4fc6-9eba-1f4256f695cc None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:48.144693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-081361f6-bceb-4fc6-9eba-1f4256f695cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:48.151052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573ed58-3720-473b-8d10-15a9b8b95c34 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:48.195778 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794a69b-1f61-4350-a726-7d45a5c98810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:48.196444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794a69b-1f61-4350-a726-7d45a5c98810 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:48.196727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794a69b-1f61-4350-a726-7d45a5c98810 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:48.197003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794a69b-1f61-4350-a726-7d45a5c98810 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:48.197848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794a69b-1f61-4350-a726-7d45a5c98810 None admin] RBAC: Adding query filter params (group.id=122d46298c4d4c6dab814b440bb0eb84, role.id=5c87e804d0764309ba08ad6e06b23a87, scope.project.id=475f061ace89472aa69923909a044007) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:48.198186 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794a69b-1f61-4350-a726-7d45a5c98810 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:48.199628 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794a69b-1f61-4350-a726-7d45a5c98810 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:48.213777 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 189/385] 10.222.0.65 () {56 vars in 1376 bytes} [Tue May 10 19:57:48 2022] GET /identity/v3/role_assignments?group.id=122d46298c4d4c6dab814b440bb0eb84&scope.project.id=475f061ace89472aa69923909a044007&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:48.214924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794a69b-1f61-4350-a726-7d45a5c98810 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:48.217220 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 197/386] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:48.261722 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081361f6-bceb-4fc6-9eba-1f4256f695cc None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:48.421287 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794a69b-1f61-4350-a726-7d45a5c98810 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:48.474955 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081361f6-bceb-4fc6-9eba-1f4256f695cc None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:48.481557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349c979-ab6a-4d5f-b6a2-0b8b467b046f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:48.482458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349c979-ab6a-4d5f-b6a2-0b8b467b046f None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:48.482743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349c979-ab6a-4d5f-b6a2-0b8b467b046f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:48.483015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349c979-ab6a-4d5f-b6a2-0b8b467b046f None admin] PATH_INFO: `/v3/groups/b119fac6c6f04c74baf3a690c852feea` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:48.484218 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349c979-ab6a-4d5f-b6a2-0b8b467b046f None admin] RBAC: Authorizing `identity:get_group(group_id=b119fac6c6f04c74baf3a690c852feea)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:48.485680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349c979-ab6a-4d5f-b6a2-0b8b467b046f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:48.487638 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 190/387] 10.222.0.65 () {56 vars in 1180 bytes} [Tue May 10 19:57:48 2022] GET /identity/v3/groups/b119fac6c6f04c74baf3a690c852feea => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:48.499080 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349c979-ab6a-4d5f-b6a2-0b8b467b046f None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:48.517823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c67e28-2d22-438f-b510-b54eaddec850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:48.518655 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c67e28-2d22-438f-b510-b54eaddec850 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:48.518933 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c67e28-2d22-438f-b510-b54eaddec850 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:48.519204 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c67e28-2d22-438f-b510-b54eaddec850 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:48.535563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c67e28-2d22-438f-b510-b54eaddec850 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:48.548210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-89c67e28-2d22-438f-b510-b54eaddec850 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:48.548210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:48.548210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:48.548210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:48.548210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:48.548210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:48.548210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:48.548210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:48.548210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:48.548210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:48.548210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:48.548210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:48.548210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:48.548210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:48.548210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:48.548210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:48.549190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:48.549190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:48.549190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:48.549190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:48.549190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:48.549190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:48.549190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:48.549190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:48.549190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:48.549190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:48.549190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:48.549190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:48.549190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:48.549190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:48.549190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:48.549190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:48.550003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:48.550003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:48.550003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:48.550003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:48.550003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:48.550003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:48.550003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:48.550003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:48.550003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:48.550003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:48.550003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:48.550003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:48.550648 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 198/388] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:57:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:48.571682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c67e28-2d22-438f-b510-b54eaddec850 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:48.615269 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349c979-ab6a-4d5f-b6a2-0b8b467b046f None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:48.654461 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f0cee-d644-4c1e-a890-b675c2096fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:48.655335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f0cee-d644-4c1e-a890-b675c2096fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:48.655618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f0cee-d644-4c1e-a890-b675c2096fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:48.655913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f0cee-d644-4c1e-a890-b675c2096fb6 None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:48.657894 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f0cee-d644-4c1e-a890-b675c2096fb6 None admin] RBAC: Authorizing `identity:get_project(project_id=675b1b490743424e9d44dd2e484dfa95)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:48.659373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f0cee-d644-4c1e-a890-b675c2096fb6 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:48.661121 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 191/389] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:48 2022] GET /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:48.668252 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f0cee-d644-4c1e-a890-b675c2096fb6 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:48.745161 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c67e28-2d22-438f-b510-b54eaddec850 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:48.817556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0bf6c-6648-47da-841b-2264f7cd2744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:48.818269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0bf6c-6648-47da-841b-2264f7cd2744 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:48.818552 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0bf6c-6648-47da-841b-2264f7cd2744 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:48.818832 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0bf6c-6648-47da-841b-2264f7cd2744 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:48.819598 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0bf6c-6648-47da-841b-2264f7cd2744 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:48.819909 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0bf6c-6648-47da-841b-2264f7cd2744 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:48.821293 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0bf6c-6648-47da-841b-2264f7cd2744 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:48.837582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 199/390] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:57:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:48.843537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0bf6c-6648-47da-841b-2264f7cd2744 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:48.868516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f0cee-d644-4c1e-a890-b675c2096fb6 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:48.952112 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e26dd-c7f5-4681-b7c6-9170ddf47d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:48.953388 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e26dd-c7f5-4681-b7c6-9170ddf47d39 None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:48.953875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e26dd-c7f5-4681-b7c6-9170ddf47d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:48.954372 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e26dd-c7f5-4681-b7c6-9170ddf47d39 None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95/groups/b119fac6c6f04c74baf3a690c852feea/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:48.960038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0bf6c-6648-47da-841b-2264f7cd2744 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:48.960604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e26dd-c7f5-4681-b7c6-9170ddf47d39 None admin] RBAC: Authorizing `identity:create_grant(project_id=675b1b490743424e9d44dd2e484dfa95, group_id=b119fac6c6f04c74baf3a690c852feea, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:48.962327 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e26dd-c7f5-4681-b7c6-9170ddf47d39 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:48.997529 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ef1fa-dc5d-4612-b27a-77a0b46c8703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:48.998341 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ef1fa-dc5d-4612-b27a-77a0b46c8703 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:48.998625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ef1fa-dc5d-4612-b27a-77a0b46c8703 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:48.998906 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ef1fa-dc5d-4612-b27a-77a0b46c8703 None admin] PATH_INFO: `/v3/groups/b119fac6c6f04c74baf3a690c852feea` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:49.001925 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ef1fa-dc5d-4612-b27a-77a0b46c8703 None admin] RBAC: Authorizing `identity:get_group(group_id=b119fac6c6f04c74baf3a690c852feea)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:49.003402 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ef1fa-dc5d-4612-b27a-77a0b46c8703 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:49.005387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 200/391] 10.222.0.65 () {56 vars in 1180 bytes} [Tue May 10 19:57:48 2022] GET /identity/v3/groups/b119fac6c6f04c74baf3a690c852feea => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:49.010969 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ef1fa-dc5d-4612-b27a-77a0b46c8703 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:49.061773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 192/392] 10.222.0.65 () {58 vars in 1442 bytes} [Tue May 10 19:57:48 2022] PUT /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95/groups/b119fac6c6f04c74baf3a690c852feea/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:57:49.122684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ef1fa-dc5d-4612-b27a-77a0b46c8703 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:49.158174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69f55c-c534-49ee-9106-4df6c9d670dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:49.159108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69f55c-c534-49ee-9106-4df6c9d670dc None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:49.159397 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69f55c-c534-49ee-9106-4df6c9d670dc None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:49.159679 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69f55c-c534-49ee-9106-4df6c9d670dc None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:49.161484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69f55c-c534-49ee-9106-4df6c9d670dc None admin] RBAC: Authorizing `identity:get_project(project_id=675b1b490743424e9d44dd2e484dfa95)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:49.163020 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69f55c-c534-49ee-9106-4df6c9d670dc None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:49.165023 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 201/393] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:49 2022] GET /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:49.170862 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e26dd-c7f5-4681-b7c6-9170ddf47d39 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:49.280901 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e26dd-c7f5-4681-b7c6-9170ddf47d39 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:49.319926 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265283a0-fd5a-4865-9dcd-63e5a5a8a5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:49.320825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265283a0-fd5a-4865-9dcd-63e5a5a8a5e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:49.321116 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265283a0-fd5a-4865-9dcd-63e5a5a8a5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:49.321393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265283a0-fd5a-4865-9dcd-63e5a5a8a5e2 None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95/groups/b119fac6c6f04c74baf3a690c852feea/roles/4fbb0a346f5f4f36bb32664ee799db42` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:49.323738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265283a0-fd5a-4865-9dcd-63e5a5a8a5e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=675b1b490743424e9d44dd2e484dfa95, group_id=b119fac6c6f04c74baf3a690c852feea, role_id=4fbb0a346f5f4f36bb32664ee799db42)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:49.325227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265283a0-fd5a-4865-9dcd-63e5a5a8a5e2 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:49.427582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 193/394] 10.222.0.65 () {58 vars in 1442 bytes} [Tue May 10 19:57:49 2022] PUT /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95/groups/b119fac6c6f04c74baf3a690c852feea/roles/4fbb0a346f5f4f36bb32664ee799db42 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:57:49.791084 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47aa4c-6919-4747-b629-b2d7cc8cad39 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:49.791595 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47aa4c-6919-4747-b629-b2d7cc8cad39 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:49.791698 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47aa4c-6919-4747-b629-b2d7cc8cad39 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:49.793253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 202/395] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:49.799207 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842f21a-8c89-41bb-a2de-888c04ea0809 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:49.799759 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842f21a-8c89-41bb-a2de-888c04ea0809 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:49.800234 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842f21a-8c89-41bb-a2de-888c04ea0809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:49.819737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-5842f21a-8c89-41bb-a2de-888c04ea0809 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:49.821275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-5842f21a-8c89-41bb-a2de-888c04ea0809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:49.897607 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 194/396] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:57:49.915173 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47aa4c-6919-4747-b629-b2d7cc8cad39 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:50.032531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47aa4c-6919-4747-b629-b2d7cc8cad39 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:50.073730 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0db90f-9a82-48e9-a417-88f243d53c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:50.074881 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0db90f-9a82-48e9-a417-88f243d53c92 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:50.075364 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0db90f-9a82-48e9-a417-88f243d53c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:50.075850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0db90f-9a82-48e9-a417-88f243d53c92 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:50.088043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0db90f-9a82-48e9-a417-88f243d53c92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:50.096289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-cd0db90f-9a82-48e9-a417-88f243d53c92 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:50.096289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:50.096289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:50.096289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:50.096289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:50.096289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:50.096289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:50.096289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:50.096289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:50.096289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:50.096289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:50.096289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:50.096289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:50.096289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:50.096289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:50.096289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:50.097157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:50.097157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:50.097157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:50.097157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:50.097157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:50.097157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:50.097157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:50.097157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:50.097157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:50.097157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:50.097157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:50.097157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:50.097157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:50.097157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:50.097157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:50.097157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:50.098028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:50.098028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:50.098028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:50.098028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:50.098028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:50.098028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:50.098028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:50.098028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:50.098028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:50.098028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:50.098028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:50.098028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:50.105057 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 203/397] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:50.106825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842f21a-8c89-41bb-a2de-888c04ea0809 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:50.225411 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62d958-12b2-4ae4-a500-d58a86dbbc28 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:50.231312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62d958-12b2-4ae4-a500-d58a86dbbc28 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:50.231312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62d958-12b2-4ae4-a500-d58a86dbbc28 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:50.231312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 204/398] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:57:50.233503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa9da6-f12a-4330-884d-1e579fe7f837 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:50.233795 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa9da6-f12a-4330-884d-1e579fe7f837 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:50.234096 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa9da6-f12a-4330-884d-1e579fe7f837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:50.235198 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842f21a-8c89-41bb-a2de-888c04ea0809 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:50.264405 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-e7fa9da6-f12a-4330-884d-1e579fe7f837 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:50.265618 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-e7fa9da6-f12a-4330-884d-1e579fe7f837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:50.280785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9883721-5efa-424b-9b40-37fc21f4f299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:50.281453 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9883721-5efa-424b-9b40-37fc21f4f299 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:50.281727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9883721-5efa-424b-9b40-37fc21f4f299 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:50.281991 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9883721-5efa-424b-9b40-37fc21f4f299 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:50.282790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9883721-5efa-424b-9b40-37fc21f4f299 None admin] RBAC: Adding query filter params (name=member) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:50.283062 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9883721-5efa-424b-9b40-37fc21f4f299 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:50.284427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9883721-5efa-424b-9b40-37fc21f4f299 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:50.303914 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 195/399] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:50.309602 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9883721-5efa-424b-9b40-37fc21f4f299 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:50.389298 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 205/400] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:57:50.436463 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa9da6-f12a-4330-884d-1e579fe7f837 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:50.442201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9883721-5efa-424b-9b40-37fc21f4f299 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:50.478975 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5f9db-fcd6-481e-9fd1-5bcd9ef030b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:50.479824 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5f9db-fcd6-481e-9fd1-5bcd9ef030b3 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:50.480099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5f9db-fcd6-481e-9fd1-5bcd9ef030b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:50.480364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5f9db-fcd6-481e-9fd1-5bcd9ef030b3 None admin] PATH_INFO: `/v3/groups/b119fac6c6f04c74baf3a690c852feea` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:50.481566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5f9db-fcd6-481e-9fd1-5bcd9ef030b3 None admin] RBAC: Authorizing `identity:get_group(group_id=b119fac6c6f04c74baf3a690c852feea)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:50.483137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5f9db-fcd6-481e-9fd1-5bcd9ef030b3 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:50.485060 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 196/401] 10.222.0.65 () {56 vars in 1180 bytes} [Tue May 10 19:57:50 2022] GET /identity/v3/groups/b119fac6c6f04c74baf3a690c852feea => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:50.491521 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5f9db-fcd6-481e-9fd1-5bcd9ef030b3 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:50.624518 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa9da6-f12a-4330-884d-1e579fe7f837 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:50.663342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354fb13-1509-4268-8426-36bc27e2c4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:50.664632 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354fb13-1509-4268-8426-36bc27e2c4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:50.664928 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354fb13-1509-4268-8426-36bc27e2c4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:50.665201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354fb13-1509-4268-8426-36bc27e2c4e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:50.679068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354fb13-1509-4268-8426-36bc27e2c4e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:50.681674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5f9db-fcd6-481e-9fd1-5bcd9ef030b3 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:50.688704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-8354fb13-1509-4268-8426-36bc27e2c4e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:50.688704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:50.688704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:50.688704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:50.688704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:50.688704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:50.688704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:50.688704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:50.688704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:50.688704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:50.688704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:50.688704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:50.688704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:50.688704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:50.688704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:50.688704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:50.689626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:50.689626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:50.689626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:50.689626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:50.689626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:50.689626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:50.689626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:50.689626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:50.689626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:50.689626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:50.689626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:50.689626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:50.689626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:50.689626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:50.689626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:50.689626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:50.691050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:50.691050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:50.691050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:50.691050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:50.691050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:50.691050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:50.691050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:50.691050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:50.691050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:50.691050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:50.691050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:50.691050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:50.691050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 206/402] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:57:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:57:50.699665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354fb13-1509-4268-8426-36bc27e2c4e6 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:50.756227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c28985-d343-4541-a546-8b40f6eb8334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:50.757845 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c28985-d343-4541-a546-8b40f6eb8334 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:50.757845 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c28985-d343-4541-a546-8b40f6eb8334 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:50.757845 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c28985-d343-4541-a546-8b40f6eb8334 None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:50.762712 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c28985-d343-4541-a546-8b40f6eb8334 None admin] RBAC: Authorizing `identity:get_project(project_id=775bb0dd255b49ca81b1452cbe667589)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:50.764179 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c28985-d343-4541-a546-8b40f6eb8334 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:50.770287 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 197/403] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:50 2022] GET /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:50.783676 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c28985-d343-4541-a546-8b40f6eb8334 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:50.831109 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354fb13-1509-4268-8426-36bc27e2c4e6 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:50.874109 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c421090-166e-42b3-86e8-869f5f289c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:50.875268 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421090-166e-42b3-86e8-869f5f289c4b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:50.875762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421090-166e-42b3-86e8-869f5f289c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:50.876240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421090-166e-42b3-86e8-869f5f289c4b None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:50.877235 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c421090-166e-42b3-86e8-869f5f289c4b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:50.877854 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c421090-166e-42b3-86e8-869f5f289c4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:50.879551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c421090-166e-42b3-86e8-869f5f289c4b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:50.889625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 207/404] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:57:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:50.896319 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c421090-166e-42b3-86e8-869f5f289c4b None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:51.015872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c28985-d343-4541-a546-8b40f6eb8334 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:51.075671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c421090-166e-42b3-86e8-869f5f289c4b None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:51.087286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf8d2c-8dd8-48c0-821a-0b39ef82afeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:51.088475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf8d2c-8dd8-48c0-821a-0b39ef82afeb None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:51.088970 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf8d2c-8dd8-48c0-821a-0b39ef82afeb None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:51.089459 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf8d2c-8dd8-48c0-821a-0b39ef82afeb None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589/groups/b119fac6c6f04c74baf3a690c852feea/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:51.092497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf8d2c-8dd8-48c0-821a-0b39ef82afeb None admin] RBAC: Authorizing `identity:create_grant(project_id=775bb0dd255b49ca81b1452cbe667589, group_id=b119fac6c6f04c74baf3a690c852feea, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:51.094186 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf8d2c-8dd8-48c0-821a-0b39ef82afeb None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:51.126556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b356b93-7379-44af-93e5-8f6e25d411e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:51.127454 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b356b93-7379-44af-93e5-8f6e25d411e4 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:51.127737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b356b93-7379-44af-93e5-8f6e25d411e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:51.128009 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b356b93-7379-44af-93e5-8f6e25d411e4 None admin] PATH_INFO: `/v3/groups/b119fac6c6f04c74baf3a690c852feea` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:51.129800 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b356b93-7379-44af-93e5-8f6e25d411e4 None admin] RBAC: Authorizing `identity:get_group(group_id=b119fac6c6f04c74baf3a690c852feea)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:51.131267 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b356b93-7379-44af-93e5-8f6e25d411e4 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:51.133223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 208/405] 10.222.0.65 () {56 vars in 1180 bytes} [Tue May 10 19:57:50 2022] GET /identity/v3/groups/b119fac6c6f04c74baf3a690c852feea => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:51.134512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73622da-b04c-48df-87a1-f7d3d898e059 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:51.134798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73622da-b04c-48df-87a1-f7d3d898e059 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:51.135094 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73622da-b04c-48df-87a1-f7d3d898e059 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:51.136520 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 209/406] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:51.138216 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73622da-b04c-48df-87a1-f7d3d898e059 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:51.246279 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73622da-b04c-48df-87a1-f7d3d898e059 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:51.274515 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 198/407] 10.222.0.65 () {58 vars in 1442 bytes} [Tue May 10 19:57:50 2022] PUT /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589/groups/b119fac6c6f04c74baf3a690c852feea/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 19:57:51.276980 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904650c1-55a0-49e5-85f2-47ba798cca64 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:51.277490 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904650c1-55a0-49e5-85f2-47ba798cca64 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:51.278498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904650c1-55a0-49e5-85f2-47ba798cca64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:51.295877 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ad61c-74bd-41fb-9c44-54225963d7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:51.296883 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ad61c-74bd-41fb-9c44-54225963d7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:51.297169 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ad61c-74bd-41fb-9c44-54225963d7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:51.297449 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ad61c-74bd-41fb-9c44-54225963d7c5 None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:51.298811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ad61c-74bd-41fb-9c44-54225963d7c5 None admin] RBAC: Authorizing `identity:get_project(project_id=775bb0dd255b49ca81b1452cbe667589)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:51.300495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ad61c-74bd-41fb-9c44-54225963d7c5 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:51.303506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 210/408] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:51 2022] GET /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:51.308755 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ad61c-74bd-41fb-9c44-54225963d7c5 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:51.317571 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-904650c1-55a0-49e5-85f2-47ba798cca64 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:51.319213 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-904650c1-55a0-49e5-85f2-47ba798cca64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:51.446006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 199/409] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:51.481404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904650c1-55a0-49e5-85f2-47ba798cca64 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:51.496049 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ad61c-74bd-41fb-9c44-54225963d7c5 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:51.566316 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49797f3b-8b22-4aa2-8d72-6833dcaa6eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:51.567679 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49797f3b-8b22-4aa2-8d72-6833dcaa6eb9 None admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:51.567964 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49797f3b-8b22-4aa2-8d72-6833dcaa6eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:51.568242 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49797f3b-8b22-4aa2-8d72-6833dcaa6eb9 None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589/groups/b119fac6c6f04c74baf3a690c852feea/roles/4fbb0a346f5f4f36bb32664ee799db42` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:51.572113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49797f3b-8b22-4aa2-8d72-6833dcaa6eb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=775bb0dd255b49ca81b1452cbe667589, group_id=b119fac6c6f04c74baf3a690c852feea, role_id=4fbb0a346f5f4f36bb32664ee799db42)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:51.573560 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49797f3b-8b22-4aa2-8d72-6833dcaa6eb9 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:51.622033 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904650c1-55a0-49e5-85f2-47ba798cca64 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:51.661258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 211/410] 10.222.0.65 () {58 vars in 1442 bytes} [Tue May 10 19:57:51 2022] PUT /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589/groups/b119fac6c6f04c74baf3a690c852feea/roles/4fbb0a346f5f4f36bb32664ee799db42 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:57:51.666962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9af66-616a-4f54-8e67-d1585ab147f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:51.667839 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9af66-616a-4f54-8e67-d1585ab147f5 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:51.668125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9af66-616a-4f54-8e67-d1585ab147f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:51.668400 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9af66-616a-4f54-8e67-d1585ab147f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:51.686781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9af66-616a-4f54-8e67-d1585ab147f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:51.695331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-f3f9af66-616a-4f54-8e67-d1585ab147f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:51.695331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:51.695331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:51.695331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:51.695331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:51.695331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:51.695331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:51.695331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:51.695331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:51.695331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:51.695331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:51.695331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:51.695331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:51.695331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:51.695331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:51.695331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:51.696240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:51.696240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:51.696240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:51.696240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:51.696240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:51.696240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:51.696240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:51.696240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:51.696240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:51.696240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:51.696240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:51.696240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:51.696240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:51.696240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:51.696240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:51.696240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:51.697656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:51.697656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:51.697656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:51.697656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:51.697656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:51.697656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:51.697656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:51.697656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:51.697656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:51.697656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:51.697656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:51.697656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:51.697656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 200/411] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:57:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:57:51.702886 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49797f3b-8b22-4aa2-8d72-6833dcaa6eb9 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:51.854788 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49797f3b-8b22-4aa2-8d72-6833dcaa6eb9 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:51.902056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f2316-a89e-47da-8d01-a499552cd42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:51.902757 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f2316-a89e-47da-8d01-a499552cd42b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:51.903045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f2316-a89e-47da-8d01-a499552cd42b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:51.903322 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f2316-a89e-47da-8d01-a499552cd42b None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:51.904088 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f2316-a89e-47da-8d01-a499552cd42b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:51.904399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f2316-a89e-47da-8d01-a499552cd42b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:51.905795 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f2316-a89e-47da-8d01-a499552cd42b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:51.915407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 212/412] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:51.921368 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9af66-616a-4f54-8e67-d1585ab147f5 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:52.050206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9af66-616a-4f54-8e67-d1585ab147f5 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:52.086682 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9b22a-4716-49b2-aeff-9b54f6b81192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:52.087599 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9b22a-4716-49b2-aeff-9b54f6b81192 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:52.087888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9b22a-4716-49b2-aeff-9b54f6b81192 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:52.088255 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9b22a-4716-49b2-aeff-9b54f6b81192 None admin] PATH_INFO: `/v3/groups/122d46298c4d4c6dab814b440bb0eb84` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:52.089478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9b22a-4716-49b2-aeff-9b54f6b81192 None admin] RBAC: Authorizing `identity:get_group(group_id=122d46298c4d4c6dab814b440bb0eb84)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:52.091508 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9b22a-4716-49b2-aeff-9b54f6b81192 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:52.093580 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 201/413] 10.222.0.65 () {56 vars in 1180 bytes} [Tue May 10 19:57:51 2022] GET /identity/v3/groups/122d46298c4d4c6dab814b440bb0eb84 => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:52.102717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f2316-a89e-47da-8d01-a499552cd42b None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:52.174157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d5586-0e50-4fc5-b721-835a7969b71a None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:52.178118 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d5586-0e50-4fc5-b721-835a7969b71a None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:52.178118 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d5586-0e50-4fc5-b721-835a7969b71a None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:52.178118 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 202/414] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:57:52.183595 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b5cd6-30ab-432d-9b6f-7a4d49282f8f None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:52.184024 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b5cd6-30ab-432d-9b6f-7a4d49282f8f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:52.184402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b5cd6-30ab-432d-9b6f-7a4d49282f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:52.208128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-af6b5cd6-30ab-432d-9b6f-7a4d49282f8f None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:52.209344 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-af6b5cd6-30ab-432d-9b6f-7a4d49282f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:52.281164 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f2316-a89e-47da-8d01-a499552cd42b None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:52.315896 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c396d61-e800-4e62-aa32-8721c9e7aa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:52.316844 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c396d61-e800-4e62-aa32-8721c9e7aa2a None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:52.317128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c396d61-e800-4e62-aa32-8721c9e7aa2a None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:52.317455 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c396d61-e800-4e62-aa32-8721c9e7aa2a None admin] PATH_INFO: `/v3/projects/475f061ace89472aa69923909a044007` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:52.318694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c396d61-e800-4e62-aa32-8721c9e7aa2a None admin] RBAC: Authorizing `identity:get_project(project_id=475f061ace89472aa69923909a044007)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:52.320158 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c396d61-e800-4e62-aa32-8721c9e7aa2a None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:52.322232 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 213/415] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:52 2022] GET /identity/v3/projects/475f061ace89472aa69923909a044007 => generated 344 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:52.328051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c396d61-e800-4e62-aa32-8721c9e7aa2a None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:52.340003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 203/416] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:57:52.397496 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df096da-fa1f-477b-a5d4-9ac646fe0b1f None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:52.397840 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df096da-fa1f-477b-a5d4-9ac646fe0b1f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:52.398058 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df096da-fa1f-477b-a5d4-9ac646fe0b1f None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:52.401759 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 204/417] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:52.403045 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df096da-fa1f-477b-a5d4-9ac646fe0b1f None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:52.460448 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c396d61-e800-4e62-aa32-8721c9e7aa2a None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:52.494415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af42d0-eada-4bda-9a2b-0654c442cf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:52.495292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af42d0-eada-4bda-9a2b-0654c442cf62 None admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:52.495573 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af42d0-eada-4bda-9a2b-0654c442cf62 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:52.495847 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af42d0-eada-4bda-9a2b-0654c442cf62 None admin] PATH_INFO: `/v3/projects/475f061ace89472aa69923909a044007/groups/122d46298c4d4c6dab814b440bb0eb84/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:52.497897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af42d0-eada-4bda-9a2b-0654c442cf62 None admin] RBAC: Authorizing `identity:create_grant(project_id=475f061ace89472aa69923909a044007, group_id=122d46298c4d4c6dab814b440bb0eb84, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:52.499365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af42d0-eada-4bda-9a2b-0654c442cf62 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:52.513664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df096da-fa1f-477b-a5d4-9ac646fe0b1f None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:52.547831 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14d190-f024-4d57-b4e7-e53e8b06040e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:52.548681 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14d190-f024-4d57-b4e7-e53e8b06040e None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:52.548961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14d190-f024-4d57-b4e7-e53e8b06040e None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:52.549275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14d190-f024-4d57-b4e7-e53e8b06040e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:52.559445 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14d190-f024-4d57-b4e7-e53e8b06040e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:52.567364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-ac14d190-f024-4d57-b4e7-e53e8b06040e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:52.567364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:52.567364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:52.567364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:52.567364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:52.567364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:52.567364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:52.567364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:52.567364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:52.567364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:52.567364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:52.567364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:52.567364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:52.567364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:52.567364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:52.567364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:52.568263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:52.568263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:52.568263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:52.568263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:52.568263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:52.568263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:52.568263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:52.568263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:52.568263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:52.568263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:52.568263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:52.568263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:52.568263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:52.568263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:52.568263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:52.568263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:52.569050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:52.569050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:52.569050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:52.569050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:52.569050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:52.569050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:52.569050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:52.569050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:52.569050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:52.569050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:52.569050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:52.569050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:52.570242 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 205/418] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:52.572101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dc290-67a3-414c-ae5e-d2eabcae7dc4 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:52.573237 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dc290-67a3-414c-ae5e-d2eabcae7dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:52.574512 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dc290-67a3-414c-ae5e-d2eabcae7dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:52.581319 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 214/419] 10.222.0.65 () {58 vars in 1442 bytes} [Tue May 10 19:57:52 2022] PUT /identity/v3/projects/475f061ace89472aa69923909a044007/groups/122d46298c4d4c6dab814b440bb0eb84/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:57:52.582966 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af42d0-eada-4bda-9a2b-0654c442cf62 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:52.594135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-d31dc290-67a3-414c-ae5e-d2eabcae7dc4 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:52.595807 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-d31dc290-67a3-414c-ae5e-d2eabcae7dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:52.730057 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af42d0-eada-4bda-9a2b-0654c442cf62 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:52.776895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 206/420] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:52.800482 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1158931-d00a-46ca-957c-6ecdf20fe61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:52.801555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1158931-d00a-46ca-957c-6ecdf20fe61b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:52.802045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1158931-d00a-46ca-957c-6ecdf20fe61b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:52.802564 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1158931-d00a-46ca-957c-6ecdf20fe61b None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:52.803527 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1158931-d00a-46ca-957c-6ecdf20fe61b None admin] RBAC: Adding query filter params (name=member) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:52.804058 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1158931-d00a-46ca-957c-6ecdf20fe61b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:52.805634 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1158931-d00a-46ca-957c-6ecdf20fe61b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:52.806784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dc290-67a3-414c-ae5e-d2eabcae7dc4 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:52.819853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 215/421] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:52.825462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1158931-d00a-46ca-957c-6ecdf20fe61b None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:52.951333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dc290-67a3-414c-ae5e-d2eabcae7dc4 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:52.985512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1158931-d00a-46ca-957c-6ecdf20fe61b None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:53.021635 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4b559-ab71-4188-a1de-b443107b42f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:53.022479 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4b559-ab71-4188-a1de-b443107b42f7 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:53.022762 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4b559-ab71-4188-a1de-b443107b42f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:53.023038 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4b559-ab71-4188-a1de-b443107b42f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:53.042024 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f4b559-ab71-4188-a1de-b443107b42f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:53.050192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-b3f4b559-ab71-4188-a1de-b443107b42f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:53.050192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:53.050192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:53.050192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:53.050192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:53.050192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:53.050192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:53.050192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:53.050192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:53.050192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:53.050192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:53.050192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:53.050192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:53.050192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:53.050192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:53.050192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:53.051690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:53.051690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:53.051690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:53.051690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:53.051690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:53.051690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:53.051690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:53.051690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:53.051690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:53.051690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:53.051690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:53.051690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:53.051690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:53.051690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:53.051690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:53.051690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:53.052502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:53.052502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:53.052502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:53.052502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:53.052502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:53.052502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:53.052502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:53.052502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:53.052502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:53.052502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:53.052502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:53.052502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:53.052502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 207/422] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:57:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:57:53.060010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a7cd1-cdb8-4f25-b8e2-571c846b50f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:53.060673 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a7cd1-cdb8-4f25-b8e2-571c846b50f3 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:53.060961 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a7cd1-cdb8-4f25-b8e2-571c846b50f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:53.061241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a7cd1-cdb8-4f25-b8e2-571c846b50f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:53.072498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4b559-ab71-4188-a1de-b443107b42f7 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:53.191143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a7cd1-cdb8-4f25-b8e2-571c846b50f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:53.192700 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a7cd1-cdb8-4f25-b8e2-571c846b50f3 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:53.221776 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4b559-ab71-4188-a1de-b443107b42f7 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:53.266362 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ced24-9f36-40d0-8a9a-013a27900fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:53.266965 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ced24-9f36-40d0-8a9a-013a27900fba None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:53.267251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ced24-9f36-40d0-8a9a-013a27900fba None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:53.267535 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ced24-9f36-40d0-8a9a-013a27900fba None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:53.268298 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ced24-9f36-40d0-8a9a-013a27900fba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:53.268608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ced24-9f36-40d0-8a9a-013a27900fba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:53.270556 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ced24-9f36-40d0-8a9a-013a27900fba None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:53.280361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 208/423] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:57:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:53.286971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ced24-9f36-40d0-8a9a-013a27900fba None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:53.356401 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 216/424] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 19:57:53.361939 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a7cd1-cdb8-4f25-b8e2-571c846b50f3 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:53.432099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ced24-9f36-40d0-8a9a-013a27900fba None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:53.497958 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a7cd1-cdb8-4f25-b8e2-571c846b50f3 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:53.511670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170833a4-78bd-423f-b318-ebaa30ecfa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:53.512334 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170833a4-78bd-423f-b318-ebaa30ecfa21 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:53.512668 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170833a4-78bd-423f-b318-ebaa30ecfa21 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:53.512951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170833a4-78bd-423f-b318-ebaa30ecfa21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:53.538606 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a71a73-a4f9-47d1-b177-1cbbc40d4ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:53.539383 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a71a73-a4f9-47d1-b177-1cbbc40d4ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:53.539665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a71a73-a4f9-47d1-b177-1cbbc40d4ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:53.539939 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a71a73-a4f9-47d1-b177-1cbbc40d4ad9 None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:53.542557 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a71a73-a4f9-47d1-b177-1cbbc40d4ad9 None admin] RBAC: Authorizing `identity:get_project(project_id=675b1b490743424e9d44dd2e484dfa95)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:53.543990 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a71a73-a4f9-47d1-b177-1cbbc40d4ad9 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:53.545846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 217/425] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:53 2022] GET /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:53.554406 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a71a73-a4f9-47d1-b177-1cbbc40d4ad9 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:53.613152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170833a4-78bd-423f-b318-ebaa30ecfa21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:53.615265 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170833a4-78bd-423f-b318-ebaa30ecfa21 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:53.713293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 209/426] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:53.720658 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170833a4-78bd-423f-b318-ebaa30ecfa21 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:53.755693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a71a73-a4f9-47d1-b177-1cbbc40d4ad9 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:53.850242 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90a416-0b32-4425-9635-ebaca20d79bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:53.851446 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90a416-0b32-4425-9635-ebaca20d79bd None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:53.851928 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90a416-0b32-4425-9635-ebaca20d79bd None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:53.852394 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90a416-0b32-4425-9635-ebaca20d79bd None admin] PATH_INFO: `/v3/groups/b119fac6c6f04c74baf3a690c852feea` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:53.853767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90a416-0b32-4425-9635-ebaca20d79bd None admin] RBAC: Authorizing `identity:get_group(group_id=b119fac6c6f04c74baf3a690c852feea)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:53.859925 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90a416-0b32-4425-9635-ebaca20d79bd None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:53.862189 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 218/427] 10.222.0.65 () {56 vars in 1180 bytes} [Tue May 10 19:57:53 2022] GET /identity/v3/groups/b119fac6c6f04c74baf3a690c852feea => generated 226 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:53.866260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170833a4-78bd-423f-b318-ebaa30ecfa21 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:53.868118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90a416-0b32-4425-9635-ebaca20d79bd None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:53.903883 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1314b8-a430-452a-93a3-3465e61387fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:53.906246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1314b8-a430-452a-93a3-3465e61387fb None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:53.907578 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1314b8-a430-452a-93a3-3465e61387fb None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:53.908066 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1314b8-a430-452a-93a3-3465e61387fb None admin] PATH_INFO: `/v3/projects/675b1b490743424e9d44dd2e484dfa95` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:53.912073 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1314b8-a430-452a-93a3-3465e61387fb None admin] RBAC: Authorizing `identity:get_project(project_id=675b1b490743424e9d44dd2e484dfa95)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:53.913646 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1314b8-a430-452a-93a3-3465e61387fb None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:53.922200 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 210/428] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:53 2022] GET /identity/v3/projects/675b1b490743424e9d44dd2e484dfa95 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:53.923305 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1314b8-a430-452a-93a3-3465e61387fb None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:54.003043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90a416-0b32-4425-9635-ebaca20d79bd None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:54.046196 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e785c-6eb0-479e-889a-2c2ac94bcccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:54.047183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e785c-6eb0-479e-889a-2c2ac94bcccb None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:54.047470 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e785c-6eb0-479e-889a-2c2ac94bcccb None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:54.047909 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e785c-6eb0-479e-889a-2c2ac94bcccb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:54.049091 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e785c-6eb0-479e-889a-2c2ac94bcccb None admin] RBAC: Adding query filter params (group.id=b119fac6c6f04c74baf3a690c852feea, role.id=5f62ce25636443f8a5add67f208afe62, scope.project.id=675b1b490743424e9d44dd2e484dfa95) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:54.049683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e785c-6eb0-479e-889a-2c2ac94bcccb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:54.051734 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e785c-6eb0-479e-889a-2c2ac94bcccb None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:54.057759 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1314b8-a430-452a-93a3-3465e61387fb None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:54.059356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 219/429] 10.222.0.65 () {56 vars in 1376 bytes} [Tue May 10 19:57:53 2022] GET /identity/v3/role_assignments?group.id=b119fac6c6f04c74baf3a690c852feea&scope.project.id=675b1b490743424e9d44dd2e484dfa95&role.id=5f62ce25636443f8a5add67f208afe62 => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:54.061678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890296b-6afe-4ef1-bee4-aaf22dfa109d None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:54.061964 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890296b-6afe-4ef1-bee4-aaf22dfa109d None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:54.062279 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890296b-6afe-4ef1-bee4-aaf22dfa109d None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:54.066195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 220/430] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:54.072181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c5cc4-2ea5-4da9-a409-3debe2d3678b None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:54.072828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c5cc4-2ea5-4da9-a409-3debe2d3678b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:54.073124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c5cc4-2ea5-4da9-a409-3debe2d3678b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:54.095624 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-089c5cc4-2ea5-4da9-a409-3debe2d3678b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:54.097343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-089c5cc4-2ea5-4da9-a409-3debe2d3678b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:54.106346 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c1410-5ff5-4dd4-81e3-1e6e92ecb901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:54.107647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c1410-5ff5-4dd4-81e3-1e6e92ecb901 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:54.107935 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c1410-5ff5-4dd4-81e3-1e6e92ecb901 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:54.108213 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c1410-5ff5-4dd4-81e3-1e6e92ecb901 None admin] PATH_INFO: `/v3/groups/b119fac6c6f04c74baf3a690c852feea` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:54.109828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c1410-5ff5-4dd4-81e3-1e6e92ecb901 None admin] RBAC: Authorizing `identity:get_group(group_id=b119fac6c6f04c74baf3a690c852feea)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:54.112044 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c1410-5ff5-4dd4-81e3-1e6e92ecb901 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:54.114824 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 211/431] 10.222.0.65 () {56 vars in 1180 bytes} [Tue May 10 19:57:53 2022] GET /identity/v3/groups/b119fac6c6f04c74baf3a690c852feea => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:54.123167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c1410-5ff5-4dd4-81e3-1e6e92ecb901 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:54.230380 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 221/432] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:54.280734 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c5cc4-2ea5-4da9-a409-3debe2d3678b None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:54.320227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c1410-5ff5-4dd4-81e3-1e6e92ecb901 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:54.391591 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6620f-c374-461f-8bf7-c0fd67351374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:54.392179 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6620f-c374-461f-8bf7-c0fd67351374 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:54.392489 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6620f-c374-461f-8bf7-c0fd67351374 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:54.392794 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6620f-c374-461f-8bf7-c0fd67351374 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:54.393847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6620f-c374-461f-8bf7-c0fd67351374 None admin] RBAC: Adding query filter params (group.id=b119fac6c6f04c74baf3a690c852feea, role.id=4fbb0a346f5f4f36bb32664ee799db42, scope.project.id=675b1b490743424e9d44dd2e484dfa95) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:54.394197 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6620f-c374-461f-8bf7-c0fd67351374 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:54.397076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6620f-c374-461f-8bf7-c0fd67351374 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:54.405609 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 212/433] 10.222.0.65 () {56 vars in 1376 bytes} [Tue May 10 19:57:54 2022] GET /identity/v3/role_assignments?group.id=b119fac6c6f04c74baf3a690c852feea&scope.project.id=675b1b490743424e9d44dd2e484dfa95&role.id=4fbb0a346f5f4f36bb32664ee799db42 => generated 611 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:54.440942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c5cc4-2ea5-4da9-a409-3debe2d3678b None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:54.476891 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81a08a-cbdd-4056-bd11-e114a4a41420 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:54.477409 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81a08a-cbdd-4056-bd11-e114a4a41420 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:54.477409 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81a08a-cbdd-4056-bd11-e114a4a41420 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:54.480631 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 213/434] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:57:54.485202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ed10e-10bb-4ea4-9241-99a32ba52d23 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:54.485507 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ed10e-10bb-4ea4-9241-99a32ba52d23 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:54.486131 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ed10e-10bb-4ea4-9241-99a32ba52d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:54.495037 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7a3ef-9e6d-4532-8c8c-753f53cc52ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:54.496648 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7a3ef-9e6d-4532-8c8c-753f53cc52ad None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:54.496648 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7a3ef-9e6d-4532-8c8c-753f53cc52ad None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:54.496648 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7a3ef-9e6d-4532-8c8c-753f53cc52ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:54.511001 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-3e5ed10e-10bb-4ea4-9241-99a32ba52d23 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:54.512235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-3e5ed10e-10bb-4ea4-9241-99a32ba52d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:54.514747 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7a3ef-9e6d-4532-8c8c-753f53cc52ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:54.525129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-d4a7a3ef-9e6d-4532-8c8c-753f53cc52ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:54.525129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:54.525129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:54.525129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:54.525129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:54.525129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:54.525129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:54.525129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:54.525129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:54.525129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:54.525129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:54.525129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:54.525129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:54.525129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:54.525129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:54.525129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:54.526123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:54.526123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:54.526123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:54.526123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:54.526123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:54.526123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:54.526123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:54.526123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:54.526123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:54.526123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:54.526123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:54.526123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:54.526123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:54.526123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:54.526123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:54.526123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:54.526949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:54.526949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:54.526949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:54.526949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:54.526949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:54.526949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:54.526949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:54.526949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:54.526949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:54.526949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:54.526949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 19:57:54.526949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:54.529877 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 222/435] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:57:54.533634 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7a3ef-9e6d-4532-8c8c-753f53cc52ad None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:54.614401 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 214/436] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:57:54.635192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ed10e-10bb-4ea4-9241-99a32ba52d23 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:54.663002 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7a3ef-9e6d-4532-8c8c-753f53cc52ad None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:54.720099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751ebad-7c0a-4e4b-a186-fb611fbd51ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:54.720778 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751ebad-7c0a-4e4b-a186-fb611fbd51ea None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:54.721063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751ebad-7c0a-4e4b-a186-fb611fbd51ea None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:54.721355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751ebad-7c0a-4e4b-a186-fb611fbd51ea None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:54.722184 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5751ebad-7c0a-4e4b-a186-fb611fbd51ea None admin] RBAC: Adding query filter params (name=member) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:54.724225 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5751ebad-7c0a-4e4b-a186-fb611fbd51ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:54.725648 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5751ebad-7c0a-4e4b-a186-fb611fbd51ea None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:54.736396 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 223/437] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:57:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:54.742887 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751ebad-7c0a-4e4b-a186-fb611fbd51ea None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:54.770044 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ed10e-10bb-4ea4-9241-99a32ba52d23 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:54.817325 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66aa101-fc3e-4487-9031-4e12fce0f1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:54.818260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66aa101-fc3e-4487-9031-4e12fce0f1ac None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:54.819428 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66aa101-fc3e-4487-9031-4e12fce0f1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:54.819428 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66aa101-fc3e-4487-9031-4e12fce0f1ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:54.833883 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66aa101-fc3e-4487-9031-4e12fce0f1ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:54.844497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-d66aa101-fc3e-4487-9031-4e12fce0f1ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:54.844497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:54.844497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:54.844497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:54.844497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:54.844497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:54.844497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:54.844497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:54.844497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:54.844497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:54.844497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:54.844497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:54.844497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:54.844497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:54.844497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:54.844497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:54.845443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:54.845443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:54.845443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:54.845443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:54.845443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:54.845443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:54.845443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:54.845443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:54.845443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:54.845443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:54.845443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:54.845443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:54.845443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:54.845443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:54.845443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:54.845443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:54.846294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:54.846294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:54.846294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:54.846294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:54.846294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:54.846294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:54.846294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:54.846294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:54.846294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:54.846294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:54.846294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:57:54.846294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:54.846294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 215/438] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:57:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:57:54.853279 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66aa101-fc3e-4487-9031-4e12fce0f1ac None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:54.873303 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751ebad-7c0a-4e4b-a186-fb611fbd51ea None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:54.923098 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f341be9-1c96-4adf-9fee-c76903a514cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:54.923678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f341be9-1c96-4adf-9fee-c76903a514cc None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:54.923962 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f341be9-1c96-4adf-9fee-c76903a514cc None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:54.924239 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f341be9-1c96-4adf-9fee-c76903a514cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:54.984033 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66aa101-fc3e-4487-9031-4e12fce0f1ac None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:55.014985 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f341be9-1c96-4adf-9fee-c76903a514cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:55.016446 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f341be9-1c96-4adf-9fee-c76903a514cc None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:55.029108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c1eaa-c06c-40b7-ac02-618687e59be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:55.029782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c1eaa-c06c-40b7-ac02-618687e59be6 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:55.030073 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c1eaa-c06c-40b7-ac02-618687e59be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:55.030375 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c1eaa-c06c-40b7-ac02-618687e59be6 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:55.031150 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c1eaa-c06c-40b7-ac02-618687e59be6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:55.031443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c1eaa-c06c-40b7-ac02-618687e59be6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:55.032891 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c1eaa-c06c-40b7-ac02-618687e59be6 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:55.045044 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 216/439] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:57:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:55.051584 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c1eaa-c06c-40b7-ac02-618687e59be6 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:55.122748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 224/440] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:57:55.128071 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f341be9-1c96-4adf-9fee-c76903a514cc None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:55.159353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c1eaa-c06c-40b7-ac02-618687e59be6 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:55.193611 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d6f50-826c-4949-9800-c91f4bc4ab9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:55.194337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d6f50-826c-4949-9800-c91f4bc4ab9d None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:55.194624 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d6f50-826c-4949-9800-c91f4bc4ab9d None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:55.194905 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d6f50-826c-4949-9800-c91f4bc4ab9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:55.229958 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f341be9-1c96-4adf-9fee-c76903a514cc None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:55.259125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d6f50-826c-4949-9800-c91f4bc4ab9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:55.260566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d6f50-826c-4949-9800-c91f4bc4ab9d None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:55.261337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071f1a5-10b5-46db-912b-4b5ca64f182f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:55.262213 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071f1a5-10b5-46db-912b-4b5ca64f182f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:55.262497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071f1a5-10b5-46db-912b-4b5ca64f182f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:55.262770 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071f1a5-10b5-46db-912b-4b5ca64f182f None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:55.263957 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071f1a5-10b5-46db-912b-4b5ca64f182f None admin] RBAC: Authorizing `identity:get_project(project_id=775bb0dd255b49ca81b1452cbe667589)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:55.265363 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071f1a5-10b5-46db-912b-4b5ca64f182f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:55.267066 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 225/441] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:55 2022] GET /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:55.270027 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29da8f-c1db-465e-8a05-a906652caec8 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:55.270358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29da8f-c1db-465e-8a05-a906652caec8 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:55.270636 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29da8f-c1db-465e-8a05-a906652caec8 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:55.271968 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 226/442] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:55.274902 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29da8f-c1db-465e-8a05-a906652caec8 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:55.355220 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 217/443] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 19:57:55.358463 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4a3a9-09ef-46d9-8e59-b4ef378200be None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:55.358979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4a3a9-09ef-46d9-8e59-b4ef378200be None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:55.359607 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4a3a9-09ef-46d9-8e59-b4ef378200be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:55.371377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29da8f-c1db-465e-8a05-a906652caec8 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:55.376866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-9ca4a3a9-09ef-46d9-8e59-b4ef378200be None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:55.378036 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-9ca4a3a9-09ef-46d9-8e59-b4ef378200be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:55.403557 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909dbb7-52d6-48fb-afae-d2bf9da0c18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:55.404495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909dbb7-52d6-48fb-afae-d2bf9da0c18b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:55.404784 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909dbb7-52d6-48fb-afae-d2bf9da0c18b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:55.405063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909dbb7-52d6-48fb-afae-d2bf9da0c18b None admin] PATH_INFO: `/v3/groups/b119fac6c6f04c74baf3a690c852feea` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:55.406275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d909dbb7-52d6-48fb-afae-d2bf9da0c18b None admin] RBAC: Authorizing `identity:get_group(group_id=b119fac6c6f04c74baf3a690c852feea)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:55.407712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d909dbb7-52d6-48fb-afae-d2bf9da0c18b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:55.409324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 227/444] 10.222.0.65 () {56 vars in 1180 bytes} [Tue May 10 19:57:55 2022] GET /identity/v3/groups/b119fac6c6f04c74baf3a690c852feea => generated 226 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:55.410110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909dbb7-52d6-48fb-afae-d2bf9da0c18b None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:55.441580 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 218/445] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:57:55.450371 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4a3a9-09ef-46d9-8e59-b4ef378200be None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:55.527749 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909dbb7-52d6-48fb-afae-d2bf9da0c18b None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:55.541594 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4a3a9-09ef-46d9-8e59-b4ef378200be None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:55.559774 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d4d6c-7fac-43e3-b1ce-fd9836f4073d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:55.560622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d4d6c-7fac-43e3-b1ce-fd9836f4073d None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:55.560908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d4d6c-7fac-43e3-b1ce-fd9836f4073d None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:55.561185 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d4d6c-7fac-43e3-b1ce-fd9836f4073d None admin] PATH_INFO: `/v3/projects/775bb0dd255b49ca81b1452cbe667589` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:55.562393 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d4d6c-7fac-43e3-b1ce-fd9836f4073d None admin] RBAC: Authorizing `identity:get_project(project_id=775bb0dd255b49ca81b1452cbe667589)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:55.563823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d4d6c-7fac-43e3-b1ce-fd9836f4073d None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:55.565508 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 228/446] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:55 2022] GET /identity/v3/projects/775bb0dd255b49ca81b1452cbe667589 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:55.566311 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d4d6c-7fac-43e3-b1ce-fd9836f4073d None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:55.573238 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7d184-22a0-433b-9802-44ca4f82a65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:55.573889 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7d184-22a0-433b-9802-44ca4f82a65d None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:55.574207 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7d184-22a0-433b-9802-44ca4f82a65d None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:55.574483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7d184-22a0-433b-9802-44ca4f82a65d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:55.575320 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7d184-22a0-433b-9802-44ca4f82a65d None admin] RBAC: Adding query filter params (group.id=b119fac6c6f04c74baf3a690c852feea, role.id=5f62ce25636443f8a5add67f208afe62, scope.project.id=775bb0dd255b49ca81b1452cbe667589) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:55.575630 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7d184-22a0-433b-9802-44ca4f82a65d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:55.577058 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7d184-22a0-433b-9802-44ca4f82a65d None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:55.583250 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 219/447] 10.222.0.65 () {56 vars in 1376 bytes} [Tue May 10 19:57:55 2022] GET /identity/v3/role_assignments?group.id=b119fac6c6f04c74baf3a690c852feea&scope.project.id=775bb0dd255b49ca81b1452cbe667589&role.id=5f62ce25636443f8a5add67f208afe62 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:55.584006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7d184-22a0-433b-9802-44ca4f82a65d None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:55.677995 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d4d6c-7fac-43e3-b1ce-fd9836f4073d None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:55.687874 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7d184-22a0-433b-9802-44ca4f82a65d None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:55.716528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1959ca-93d3-4494-ad04-0253b82c81e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:55.717241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1959ca-93d3-4494-ad04-0253b82c81e0 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:55.717534 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1959ca-93d3-4494-ad04-0253b82c81e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:55.717793 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1959ca-93d3-4494-ad04-0253b82c81e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:55.726355 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54747d66-b660-43bf-82c1-8193a3a0c254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:55.727222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54747d66-b660-43bf-82c1-8193a3a0c254 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:55.727500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54747d66-b660-43bf-82c1-8193a3a0c254 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:55.727773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54747d66-b660-43bf-82c1-8193a3a0c254 None admin] PATH_INFO: `/v3/groups/b119fac6c6f04c74baf3a690c852feea` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:55.728853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1959ca-93d3-4494-ad04-0253b82c81e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:57:55.729069 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54747d66-b660-43bf-82c1-8193a3a0c254 None admin] RBAC: Authorizing `identity:get_group(group_id=b119fac6c6f04c74baf3a690c852feea)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:55.730575 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54747d66-b660-43bf-82c1-8193a3a0c254 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:55.732447 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 220/448] 10.222.0.65 () {56 vars in 1180 bytes} [Tue May 10 19:57:55 2022] GET /identity/v3/groups/b119fac6c6f04c74baf3a690c852feea => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:55.736693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-5f1959ca-93d3-4494-ad04-0253b82c81e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:55.736693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:55.736693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:55.736693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:55.736693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:55.736693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:55.736693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:55.736693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:55.736693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:55.736693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:55.736693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:55.736693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:55.736693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:57:55.736693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:57:55.736693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:57:55.736693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:57:55.737584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:57:55.737584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:57:55.737584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:57:55.737584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:57:55.737584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:55.737584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:55.737584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:55.737584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:55.737584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:55.737584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:55.737584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:55.737584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:57:55.737584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:55.737584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:55.737584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:55.737584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:55.738898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:55.738898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:55.738898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:55.738898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:55.738898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:57:55.738898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:57:55.738898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:57:55.738898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:57:55.738898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:57:55.738898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:57:55.738898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:57:55.738898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:57:55.739497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54747d66-b660-43bf-82c1-8193a3a0c254 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:55.739567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 229/449] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:57:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:55.744411 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1959ca-93d3-4494-ad04-0253b82c81e0 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:55.846272 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54747d66-b660-43bf-82c1-8193a3a0c254 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:55.857091 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1959ca-93d3-4494-ad04-0253b82c81e0 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:55.884607 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3feb0d-dea6-4ee4-878e-b7fdc99fd277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:55.885595 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3feb0d-dea6-4ee4-878e-b7fdc99fd277 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:55.885870 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3feb0d-dea6-4ee4-878e-b7fdc99fd277 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:55.886954 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3feb0d-dea6-4ee4-878e-b7fdc99fd277 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:55.887473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3feb0d-dea6-4ee4-878e-b7fdc99fd277 None admin] RBAC: Adding query filter params (group.id=b119fac6c6f04c74baf3a690c852feea, role.id=4fbb0a346f5f4f36bb32664ee799db42, scope.project.id=775bb0dd255b49ca81b1452cbe667589) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:55.887788 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3feb0d-dea6-4ee4-878e-b7fdc99fd277 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:55.889537 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3feb0d-dea6-4ee4-878e-b7fdc99fd277 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:55.893295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a68fe-f680-47c3-a1b3-3e8118ce295a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:55.895175 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a68fe-f680-47c3-a1b3-3e8118ce295a None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:55.895466 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a68fe-f680-47c3-a1b3-3e8118ce295a None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:55.895738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a68fe-f680-47c3-a1b3-3e8118ce295a None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:55.896490 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a68fe-f680-47c3-a1b3-3e8118ce295a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:55.896792 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a68fe-f680-47c3-a1b3-3e8118ce295a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:55.897435 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 221/450] 10.222.0.65 () {56 vars in 1376 bytes} [Tue May 10 19:57:55 2022] GET /identity/v3/role_assignments?group.id=b119fac6c6f04c74baf3a690c852feea&scope.project.id=775bb0dd255b49ca81b1452cbe667589&role.id=4fbb0a346f5f4f36bb32664ee799db42 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:55.898180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a68fe-f680-47c3-a1b3-3e8118ce295a None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:55.910120 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 230/451] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:57:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:57:55.913289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3feb0d-dea6-4ee4-878e-b7fdc99fd277 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:56.021834 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3feb0d-dea6-4ee4-878e-b7fdc99fd277 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:56.056448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c345d-85d2-4af8-b7fe-f0d1191d45af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:56.057104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c345d-85d2-4af8-b7fe-f0d1191d45af None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:56.057382 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c345d-85d2-4af8-b7fe-f0d1191d45af None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:56.057654 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c345d-85d2-4af8-b7fe-f0d1191d45af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:56.124312 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c345d-85d2-4af8-b7fe-f0d1191d45af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:56.125779 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c345d-85d2-4af8-b7fe-f0d1191d45af None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:56.221458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 222/452] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:57:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 19:57:56.227378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a68fe-f680-47c3-a1b3-3e8118ce295a None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:56.325800 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a68fe-f680-47c3-a1b3-3e8118ce295a None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:56.357856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e17e6-693f-4235-87a9-15c27b1b95f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:56.358737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e17e6-693f-4235-87a9-15c27b1b95f5 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:56.359025 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e17e6-693f-4235-87a9-15c27b1b95f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:56.359336 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e17e6-693f-4235-87a9-15c27b1b95f5 None admin] PATH_INFO: `/v3/projects/475f061ace89472aa69923909a044007` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:56.360523 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e17e6-693f-4235-87a9-15c27b1b95f5 None admin] RBAC: Authorizing `identity:get_project(project_id=475f061ace89472aa69923909a044007)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:56.361964 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e17e6-693f-4235-87a9-15c27b1b95f5 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:56.364293 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 231/453] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:57:56 2022] GET /identity/v3/projects/475f061ace89472aa69923909a044007 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:56.370108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c345d-85d2-4af8-b7fe-f0d1191d45af None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:56.461277 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c345d-85d2-4af8-b7fe-f0d1191d45af None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:56.493336 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1b5dc-8347-487f-8a51-d35ccbc36a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:56.494188 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1b5dc-8347-487f-8a51-d35ccbc36a4c None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:56.494469 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1b5dc-8347-487f-8a51-d35ccbc36a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:56.494740 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1b5dc-8347-487f-8a51-d35ccbc36a4c None admin] PATH_INFO: `/v3/groups/122d46298c4d4c6dab814b440bb0eb84` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:56.495879 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1b5dc-8347-487f-8a51-d35ccbc36a4c None admin] RBAC: Authorizing `identity:get_group(group_id=122d46298c4d4c6dab814b440bb0eb84)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:56.497297 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1b5dc-8347-487f-8a51-d35ccbc36a4c None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:56.498900 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 223/454] 10.222.0.65 () {56 vars in 1180 bytes} [Tue May 10 19:57:56 2022] GET /identity/v3/groups/122d46298c4d4c6dab814b440bb0eb84 => generated 229 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:56.504628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e17e6-693f-4235-87a9-15c27b1b95f5 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:56.605581 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e17e6-693f-4235-87a9-15c27b1b95f5 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:56.636741 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ea800-642c-4149-920b-cb0762b2f7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:56.637372 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ea800-642c-4149-920b-cb0762b2f7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:56.637775 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ea800-642c-4149-920b-cb0762b2f7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:56.638049 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ea800-642c-4149-920b-cb0762b2f7c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:56.638899 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ea800-642c-4149-920b-cb0762b2f7c1 None admin] RBAC: Adding query filter params (group.id=122d46298c4d4c6dab814b440bb0eb84, role.id=5c87e804d0764309ba08ad6e06b23a87, scope.project.id=475f061ace89472aa69923909a044007) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:56.639206 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ea800-642c-4149-920b-cb0762b2f7c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:56.640633 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ea800-642c-4149-920b-cb0762b2f7c1 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:56.646124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 232/455] 10.222.0.65 () {56 vars in 1376 bytes} [Tue May 10 19:57:56 2022] GET /identity/v3/role_assignments?group.id=122d46298c4d4c6dab814b440bb0eb84&scope.project.id=475f061ace89472aa69923909a044007&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:59.341289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e25c94-8ff3-4bc4-815b-5eea05c7e52f None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:59.341843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e25c94-8ff3-4bc4-815b-5eea05c7e52f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:59.341843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e25c94-8ff3-4bc4-815b-5eea05c7e52f None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:59.343614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 224/456] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:59.349087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee099413-03c5-46ab-8fb9-6e68cbde2cbb None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:59.349472 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee099413-03c5-46ab-8fb9-6e68cbde2cbb None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:59.355910 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee099413-03c5-46ab-8fb9-6e68cbde2cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:59.371090 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-ee099413-03c5-46ab-8fb9-6e68cbde2cbb None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:59.372283 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-ee099413-03c5-46ab-8fb9-6e68cbde2cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:59.464867 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 233/457] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:57:59.513340 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e25c94-8ff3-4bc4-815b-5eea05c7e52f None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:59.644487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a0d79-e9ed-4a45-8f4a-ec7f54ad7e81 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:59.645345 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a0d79-e9ed-4a45-8f4a-ec7f54ad7e81 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:59.645950 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a0d79-e9ed-4a45-8f4a-ec7f54ad7e81 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:59.646505 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e25c94-8ff3-4bc4-815b-5eea05c7e52f None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:59.648220 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 234/458] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:59.654187 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01f16e-a52a-40b8-882c-bf36547f39c1 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:59.654803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01f16e-a52a-40b8-882c-bf36547f39c1 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:59.655279 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01f16e-a52a-40b8-882c-bf36547f39c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:59.673316 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-4e01f16e-a52a-40b8-882c-bf36547f39c1 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:57:59.674898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4e01f16e-a52a-40b8-882c-bf36547f39c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:57:59.707860 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54070794-c57f-4592-9c2c-a32c889f8746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:59.708659 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54070794-c57f-4592-9c2c-a32c889f8746 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:59.708939 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54070794-c57f-4592-9c2c-a32c889f8746 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:59.709210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54070794-c57f-4592-9c2c-a32c889f8746 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:59.729846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54070794-c57f-4592-9c2c-a32c889f8746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:59.731412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54070794-c57f-4592-9c2c-a32c889f8746 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:59.746817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 235/459] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:57:59.759661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-54070794-c57f-4592-9c2c-a32c889f8746 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:57:59.759661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:59.759661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:59.759661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:59.759661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:59.759661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:59.759661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:59.759661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:59.759661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:59.759661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:59.759661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:59.759661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:59.759661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:57:59.759661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:57:59.759661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:57:59.760592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:57:59.760592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:59.760592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:59.760592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:59.760592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:59.760592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:59.760592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:59.760592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:59.760592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:59.760592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:59.760592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:59.760592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:59.760592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:59.760592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:59.760592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:59.760592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:59.760592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:59.761425 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:57:59.761425 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:57:59.761425 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:57:59.761425 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:59.761425 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 225/460] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:57:59.767353 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01f16e-a52a-40b8-882c-bf36547f39c1 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:59.791233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54070794-c57f-4592-9c2c-a32c889f8746 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:59.883330 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01f16e-a52a-40b8-882c-bf36547f39c1 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:59.912914 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54070794-c57f-4592-9c2c-a32c889f8746 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:57:59.918669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f22627-3c99-467e-83c5-c67fdedc127f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:59.919566 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f22627-3c99-467e-83c5-c67fdedc127f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:59.920050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f22627-3c99-467e-83c5-c67fdedc127f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:59.920523 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f22627-3c99-467e-83c5-c67fdedc127f None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:59.921483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f22627-3c99-467e-83c5-c67fdedc127f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:57:59.921985 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f22627-3c99-467e-83c5-c67fdedc127f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:59.923606 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f22627-3c99-467e-83c5-c67fdedc127f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:59.935579 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 236/461] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:57:59.937230 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46584f2-c984-4c63-a940-87524b6427f4 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:59.937746 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46584f2-c984-4c63-a940-87524b6427f4 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:59.938270 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46584f2-c984-4c63-a940-87524b6427f4 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:59.939927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 237/462] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:57:59.940799 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46584f2-c984-4c63-a940-87524b6427f4 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:57:59.947688 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c6089-d56d-45bf-9e00-faba255b176e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:57:59.948791 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c6089-d56d-45bf-9e00-faba255b176e None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:59.949267 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c6089-d56d-45bf-9e00-faba255b176e None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:59.949744 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c6089-d56d-45bf-9e00-faba255b176e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:57:59.964772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8c6089-d56d-45bf-9e00-faba255b176e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:57:59.966862 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8c6089-d56d-45bf-9e00-faba255b176e None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:57:59.991093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-cb8c6089-d56d-45bf-9e00-faba255b176e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:57:59.991093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:57:59.991093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:57:59.991093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:57:59.991093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:57:59.991093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:57:59.991093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:57:59.991093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:57:59.991093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:57:59.991093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:57:59.991093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:57:59.991093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:57:59.991093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:57:59.991093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:57:59.991093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:57:59.992051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:57:59.992051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:57:59.992051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:57:59.992051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:57:59.992051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:57:59.992051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:57:59.992051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:57:59.992051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:57:59.992051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:57:59.992051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:57:59.992051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:57:59.992051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:57:59.992051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:57:59.992051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:57:59.992051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:57:59.992051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:57:59.992051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:57:59.992896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:57:59.992896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:57:59.992896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:57:59.992896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:57:59.992896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 226/463] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:57:59.993746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ae393-046b-4aab-9dd3-73128b6ee286 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:57:59.994029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ae393-046b-4aab-9dd3-73128b6ee286 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:57:59.994334 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ae393-046b-4aab-9dd3-73128b6ee286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:00.011395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-755ae393-046b-4aab-9dd3-73128b6ee286 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:00.012572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-755ae393-046b-4aab-9dd3-73128b6ee286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:00.051865 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46584f2-c984-4c63-a940-87524b6427f4 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:00.079089 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 227/464] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:00.087901 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bc066-54bf-4fac-a68c-b8e798b62ec9 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:00.088511 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bc066-54bf-4fac-a68c-b8e798b62ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:00.088861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bc066-54bf-4fac-a68c-b8e798b62ec9 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:00.090688 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 228/465] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:00 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:00.091563 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bc066-54bf-4fac-a68c-b8e798b62ec9 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:00.093141 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba0e52-f482-4620-b8f9-c091f9c95d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:00.094009 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba0e52-f482-4620-b8f9-c091f9c95d2d None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:00.094645 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba0e52-f482-4620-b8f9-c091f9c95d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:00.095919 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba0e52-f482-4620-b8f9-c091f9c95d2d None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:00.096850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba0e52-f482-4620-b8f9-c091f9c95d2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:00.099095 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba0e52-f482-4620-b8f9-c091f9c95d2d None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:00.188978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bc066-54bf-4fac-a68c-b8e798b62ec9 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:00.209429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 238/466] 10.222.0.65 () {60 vars in 1132 bytes} [Tue May 10 19:57:59 2022] POST /identity/v3/users => generated 246 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:58:00.211013 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92332a84-da50-48e9-bc6c-0726f4be6ca8 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:00.211122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92332a84-da50-48e9-bc6c-0726f4be6ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:00.211355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92332a84-da50-48e9-bc6c-0726f4be6ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:00.226947 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc97cd1-f638-4699-8d73-d0a005477dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:00.227952 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc97cd1-f638-4699-8d73-d0a005477dea None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:00.228418 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc97cd1-f638-4699-8d73-d0a005477dea None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:00.228893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc97cd1-f638-4699-8d73-d0a005477dea None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:00.229824 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc97cd1-f638-4699-8d73-d0a005477dea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:00.230389 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc97cd1-f638-4699-8d73-d0a005477dea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:00.232035 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc97cd1-f638-4699-8d73-d0a005477dea None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:00.236108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-92332a84-da50-48e9-bc6c-0726f4be6ca8 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:00.245107 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-92332a84-da50-48e9-bc6c-0726f4be6ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:00.254625 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 229/467] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:00.255869 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc97cd1-f638-4699-8d73-d0a005477dea None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:00.330122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 239/468] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:00.331082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92332a84-da50-48e9-bc6c-0726f4be6ca8 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:00.367965 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc97cd1-f638-4699-8d73-d0a005477dea None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:00.406896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0645dc-9a36-42ab-a97d-c800e54383f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:00.408020 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0645dc-9a36-42ab-a97d-c800e54383f3 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:00.408532 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0645dc-9a36-42ab-a97d-c800e54383f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:00.409014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0645dc-9a36-42ab-a97d-c800e54383f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:00.420921 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0645dc-9a36-42ab-a97d-c800e54383f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:00.422825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0645dc-9a36-42ab-a97d-c800e54383f3 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:00.443614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-1d0645dc-9a36-42ab-a97d-c800e54383f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:00.443614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:00.443614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:00.443614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:00.443614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:00.443614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:00.443614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:00.443614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:00.443614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:00.443614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:00.443614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:00.443614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:00.443614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:00.443614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:00.443614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:00.444475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:00.444475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:00.444475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:00.444475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:00.444475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:00.444475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:00.444475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:00.444475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:00.444475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:00.444475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:00.444475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:00.444475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:00.444475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:00.444475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:00.444475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:00.444475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:00.444475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:00.445774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:00.445774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:00.445774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:00.445774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:00.445774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 230/469] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:00.446285 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92332a84-da50-48e9-bc6c-0726f4be6ca8 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:00.446700 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0645dc-9a36-42ab-a97d-c800e54383f3 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:00.481932 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8420aa-119a-46f5-ac8f-432d82599141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:00.482731 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8420aa-119a-46f5-ac8f-432d82599141 None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:00.483027 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8420aa-119a-46f5-ac8f-432d82599141 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:00.483390 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8420aa-119a-46f5-ac8f-432d82599141 None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:00.484117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8420aa-119a-46f5-ac8f-432d82599141 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:00.485578 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8420aa-119a-46f5-ac8f-432d82599141 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:00.553456 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0645dc-9a36-42ab-a97d-c800e54383f3 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:00.566764 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 240/470] 10.222.0.65 () {60 vars in 1132 bytes} [Tue May 10 19:58:00 2022] POST /identity/v3/users => generated 248 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:58:00.568115 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8420aa-119a-46f5-ac8f-432d82599141 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:00.589326 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6c91f-948f-4c6a-a599-cce87ea2c45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:00.590181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6c91f-948f-4c6a-a599-cce87ea2c45f None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:00.590506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6c91f-948f-4c6a-a599-cce87ea2c45f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:00.590792 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6c91f-948f-4c6a-a599-cce87ea2c45f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:00.601774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6c91f-948f-4c6a-a599-cce87ea2c45f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:00.606138 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6c91f-948f-4c6a-a599-cce87ea2c45f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:00.622522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-09c6c91f-948f-4c6a-a599-cce87ea2c45f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:00.622522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:00.622522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:00.622522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:00.622522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:00.622522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:00.622522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:00.622522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:00.622522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:00.622522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:00.622522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:00.622522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:00.622522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:00.622522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:00.622522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:00.623509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:00.623509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:00.623509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:00.623509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:00.623509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:00.623509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:00.623509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:00.623509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:00.623509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:00.623509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:00.623509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:00.623509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:00.623509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:00.623509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:00.623509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:00.623509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:00.623509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:00.624978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:00.624978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:00.624978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:00.624978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:00.624978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 231/471] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:00.632428 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6c91f-948f-4c6a-a599-cce87ea2c45f None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:00.723405 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8420aa-119a-46f5-ac8f-432d82599141 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:00.737351 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6c91f-948f-4c6a-a599-cce87ea2c45f None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:00.761669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a774824-2b28-4937-96b6-7a5f05ce3901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:00.762753 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a774824-2b28-4937-96b6-7a5f05ce3901 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:00.763245 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a774824-2b28-4937-96b6-7a5f05ce3901 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:00.763720 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a774824-2b28-4937-96b6-7a5f05ce3901 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:00.764729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a774824-2b28-4937-96b6-7a5f05ce3901 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:00.765303 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a774824-2b28-4937-96b6-7a5f05ce3901 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:00.766985 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a774824-2b28-4937-96b6-7a5f05ce3901 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:00.785325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 241/472] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:00.787153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a774824-2b28-4937-96b6-7a5f05ce3901 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:00.792516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c720b-84fe-442d-bc4d-5300631e5d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:00.793207 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c720b-84fe-442d-bc4d-5300631e5d3a None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:00.793499 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c720b-84fe-442d-bc4d-5300631e5d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:00.793775 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c720b-84fe-442d-bc4d-5300631e5d3a None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:00.794550 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c720b-84fe-442d-bc4d-5300631e5d3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:00.794860 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c720b-84fe-442d-bc4d-5300631e5d3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:00.796301 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c720b-84fe-442d-bc4d-5300631e5d3a None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:00.818822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 232/473] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:00.824354 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c720b-84fe-442d-bc4d-5300631e5d3a None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:00.951362 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a774824-2b28-4937-96b6-7a5f05ce3901 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:00.966421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c720b-84fe-442d-bc4d-5300631e5d3a None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:01.001178 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8bf29-8ba2-47c8-907e-fea1be440465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:01.001940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8bf29-8ba2-47c8-907e-fea1be440465 None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:01.002359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8bf29-8ba2-47c8-907e-fea1be440465 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:01.002733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8bf29-8ba2-47c8-907e-fea1be440465 None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:01.003595 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8bf29-8ba2-47c8-907e-fea1be440465 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:01.005326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8bf29-8ba2-47c8-907e-fea1be440465 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:01.007428 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7127452-b1a5-4147-a522-c25645b5732c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:01.008246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7127452-b1a5-4147-a522-c25645b5732c None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:01.008684 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7127452-b1a5-4147-a522-c25645b5732c None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:01.009096 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7127452-b1a5-4147-a522-c25645b5732c None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:01.009954 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7127452-b1a5-4147-a522-c25645b5732c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:01.012135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7127452-b1a5-4147-a522-c25645b5732c None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:01.126574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 242/474] 10.222.0.65 () {60 vars in 1132 bytes} [Tue May 10 19:58:00 2022] POST /identity/v3/users => generated 248 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:58:01.127924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee1f92-9f75-4e74-b889-40526918265a None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:01.128305 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee1f92-9f75-4e74-b889-40526918265a None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:01.128642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee1f92-9f75-4e74-b889-40526918265a None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:01.130142 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 243/475] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:01.135928 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4baee-42bc-4fb8-9891-5b659e80d027 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:01.136373 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4baee-42bc-4fb8-9891-5b659e80d027 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:01.136755 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4baee-42bc-4fb8-9891-5b659e80d027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:01.140559 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 233/476] 10.222.0.65 () {60 vars in 1132 bytes} [Tue May 10 19:58:00 2022] POST /identity/v3/users => generated 249 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 19:58:01.167374 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-c9d4baee-42bc-4fb8-9891-5b659e80d027 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:01.168791 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-c9d4baee-42bc-4fb8-9891-5b659e80d027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:01.248256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 244/477] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:58:01.272897 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7127452-b1a5-4147-a522-c25645b5732c None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:01.401727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7127452-b1a5-4147-a522-c25645b5732c None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:01.438910 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca711466-c6a4-40b1-ad3e-6fe702875b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:01.439626 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca711466-c6a4-40b1-ad3e-6fe702875b49 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:01.439952 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca711466-c6a4-40b1-ad3e-6fe702875b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:01.440169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca711466-c6a4-40b1-ad3e-6fe702875b49 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:01.441002 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca711466-c6a4-40b1-ad3e-6fe702875b49 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:01.442505 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca711466-c6a4-40b1-ad3e-6fe702875b49 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:01.480422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.sql.core [None req-ca711466-c6a4-40b1-ad3e-6fe702875b49 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 10 19:58:01.480422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 10 19:58:01.480422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [parameters: {'id': '1d5e815bd078465b90e246c193ad078d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 10 19:58:01.480422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80516) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 10 19:58:01.497486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-ca711466-c6a4-40b1-ad3e-6fe702875b49 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 10 19:58:01.497486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:01.497486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 10 19:58:01.497486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self.dialect.do_execute( May 10 19:58:01.497486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 10 19:58:01.497486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 10 19:58:01.497486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 10 19:58:01.497486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application result = self._query(query) May 10 19:58:01.497486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 10 19:58:01.497486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application conn.query(q) May 10 19:58:01.497486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 10 19:58:01.497486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 10 19:58:01.497486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 10 19:58:01.497486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application result.read() May 10 19:58:01.497486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 10 19:58:01.498448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 10 19:58:01.498448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 10 19:58:01.498448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application packet.raise_for_error() May 10 19:58:01.498448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 10 19:58:01.498448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 10 19:58:01.498448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 10 19:58:01.498448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 10 19:58:01.498448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 10 19:58:01.498448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:01.498448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 10 19:58:01.498448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:01.498448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:01.498448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 10 19:58:01.498448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 10 19:58:01.498448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 10 19:58:01.498448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return ref.to_dict() May 10 19:58:01.498448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 10 19:58:01.498448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application next(self.gen) May 10 19:58:01.499983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 10 19:58:01.499983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application yield resource May 10 19:58:01.499983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 10 19:58:01.499983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application next(self.gen) May 10 19:58:01.499983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 10 19:58:01.499983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self.session.rollback() May 10 19:58:01.499983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 10 19:58:01.499983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self.force_reraise() May 10 19:58:01.499983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 10 19:58:01.499983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise self.value May 10 19:58:01.499983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 10 19:58:01.499983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 10 19:58:01.499983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 10 19:58:01.499983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application session.commit() May 10 19:58:01.499983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 10 19:58:01.499983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 10 19:58:01.499983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 10 19:58:01.500843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._prepare_impl() May 10 19:58:01.500843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 10 19:58:01.500843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self.session.flush() May 10 19:58:01.500843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 10 19:58:01.500843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._flush(objects) May 10 19:58:01.500843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 10 19:58:01.500843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 10 19:58:01.500843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 10 19:58:01.500843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application compat.raise_( May 10 19:58:01.500843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 10 19:58:01.500843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception May 10 19:58:01.500843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 10 19:58:01.500843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application flush_context.execute() May 10 19:58:01.500843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 10 19:58:01.500843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rec.execute(self) May 10 19:58:01.500843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 10 19:58:01.500843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 10 19:58:01.501706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 10 19:58:01.501706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application _emit_insert_statements( May 10 19:58:01.501706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 10 19:58:01.501706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application c = connection._execute_20( May 10 19:58:01.501706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 10 19:58:01.501706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 10 19:58:01.501706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 10 19:58:01.501706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 10 19:58:01.501706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 10 19:58:01.501706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ret = self._execute_context( May 10 19:58:01.501706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 10 19:58:01.501706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 10 19:58:01.501706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 10 19:58:01.501706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 10 19:58:01.501706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 10 19:58:01.501706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception May 10 19:58:01.503769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 10 19:58:01.503769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self.dialect.do_execute( May 10 19:58:01.503769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 10 19:58:01.503769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 10 19:58:01.503769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 10 19:58:01.503769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application result = self._query(query) May 10 19:58:01.503769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 10 19:58:01.503769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application conn.query(q) May 10 19:58:01.503769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 10 19:58:01.503769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 10 19:58:01.503769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 10 19:58:01.503769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application result.read() May 10 19:58:01.503769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 10 19:58:01.503769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 10 19:58:01.503769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 10 19:58:01.503769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application packet.raise_for_error() May 10 19:58:01.505773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 10 19:58:01.505773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 10 19:58:01.505773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 10 19:58:01.505773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 10 19:58:01.505773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 10 19:58:01.505773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 10 19:58:01.505773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [parameters: {'id': '1d5e815bd078465b90e246c193ad078d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 10 19:58:01.505773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 10 19:58:01.505773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:01.505773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:01.505773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:01.505773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:01.505773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:01.505773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:01.505773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:01.505773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:01.507718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:01.507718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:01.507718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:01.507718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:01.507718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:01.507718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:01.507718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 10 19:58:01.507718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 10 19:58:01.507718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:01.507718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:01.507718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 10 19:58:01.507718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 10 19:58:01.507718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 10 19:58:01.507718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 10 19:58:01.507718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 10 19:58:01.507718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:01.510405 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 234/478] 10.222.0.65 () {60 vars in 1131 bytes} [Tue May 10 19:58:01 2022] POST /identity/v3/roles => generated 144 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:01.510560 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4baee-42bc-4fb8-9891-5b659e80d027 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:01.614882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4baee-42bc-4fb8-9891-5b659e80d027 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:01.650155 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fda4d4-80ae-4785-a819-677632acd997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:01.650969 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fda4d4-80ae-4785-a819-677632acd997 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:01.651252 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fda4d4-80ae-4785-a819-677632acd997 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:01.651528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fda4d4-80ae-4785-a819-677632acd997 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:01.662069 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fda4d4-80ae-4785-a819-677632acd997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:01.670097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-65fda4d4-80ae-4785-a819-677632acd997 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:58:01.670097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:01.670097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:01.670097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:01.670097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:01.670097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:01.670097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:01.670097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:01.670097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:01.670097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:01.670097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:01.670097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:01.670097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:01.670097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:01.670097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:01.670097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:01.672004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:01.672004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:01.672004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:01.672004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:01.672004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:01.672004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:01.672004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:01.672004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:01.672004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:01.672004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:01.672004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:01.672004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:01.672004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:01.672004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:01.672004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:01.672004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:01.672997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:01.672997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:01.672997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:01.672997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:01.672997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:01.672997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:01.672997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:01.672997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:01.672997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:01.672997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:01.672997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 19:58:01.672997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:01.672997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 245/479] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:58:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:01.679592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca711466-c6a4-40b1-ad3e-6fe702875b49 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:01.797022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca711466-c6a4-40b1-ad3e-6fe702875b49 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:01.831172 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe497362-66d4-43e2-a99c-913cfc5dccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:01.832408 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe497362-66d4-43e2-a99c-913cfc5dccaa None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:01.832690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe497362-66d4-43e2-a99c-913cfc5dccaa None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:01.832963 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe497362-66d4-43e2-a99c-913cfc5dccaa None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:01.833716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe497362-66d4-43e2-a99c-913cfc5dccaa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:01.834023 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe497362-66d4-43e2-a99c-913cfc5dccaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:01.835425 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe497362-66d4-43e2-a99c-913cfc5dccaa None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:01.844036 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 235/480] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:02.743614 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86c96f-a361-47a4-b6ba-2e65eddcc558 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:02.744199 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86c96f-a361-47a4-b6ba-2e65eddcc558 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:02.744199 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86c96f-a361-47a4-b6ba-2e65eddcc558 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:02.745757 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 246/481] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:02.751279 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a63008-203a-469a-88e5-62ee6e955762 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:02.751689 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a63008-203a-469a-88e5-62ee6e955762 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:02.751833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a63008-203a-469a-88e5-62ee6e955762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:02.769546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-b1a63008-203a-469a-88e5-62ee6e955762 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:02.770750 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b1a63008-203a-469a-88e5-62ee6e955762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:02.865891 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 236/482] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:02.889413 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86c96f-a361-47a4-b6ba-2e65eddcc558 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:03.016651 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86c96f-a361-47a4-b6ba-2e65eddcc558 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:03.049644 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69693755-2e82-495b-bb64-b0ac9197a4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:03.050985 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69693755-2e82-495b-bb64-b0ac9197a4af None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:03.051264 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69693755-2e82-495b-bb64-b0ac9197a4af None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:03.051535 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69693755-2e82-495b-bb64-b0ac9197a4af None admin] PATH_INFO: `/v3/roles/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:03.062190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69693755-2e82-495b-bb64-b0ac9197a4af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:03.070666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-69693755-2e82-495b-bb64-b0ac9197a4af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:03.070666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:03.070666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:03.070666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:03.070666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:03.070666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:03.070666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:03.070666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:03.070666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:03.070666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:03.070666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:03.070666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:03.070666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:03.070666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:03.070666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:03.070666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:03.071563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:03.071563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:03.071563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:03.071563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:03.071563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:03.071563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:03.071563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:03.071563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:03.071563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:03.071563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:03.071563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:03.071563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:03.071563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:03.071563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:03.071563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:03.071563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:03.072358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:03.072358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:03.072358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:03.072358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:03.072358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:03.072358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:03.072358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:03.072358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:03.072358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:03.072358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:03.072358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:03.072358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:03.073534 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 247/483] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:03.078829 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a63008-203a-469a-88e5-62ee6e955762 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:03.212001 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a63008-203a-469a-88e5-62ee6e955762 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:03.261206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a447a3a-55c9-441d-af52-4688ea24f002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:03.261885 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a447a3a-55c9-441d-af52-4688ea24f002 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:03.262197 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a447a3a-55c9-441d-af52-4688ea24f002 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:03.262476 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a447a3a-55c9-441d-af52-4688ea24f002 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:03.263297 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a447a3a-55c9-441d-af52-4688ea24f002 None admin] RBAC: Adding query filter params (name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:03.263610 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a447a3a-55c9-441d-af52-4688ea24f002 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:03.264987 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a447a3a-55c9-441d-af52-4688ea24f002 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:03.273922 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 237/484] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 19:58:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:03.279070 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69693755-2e82-495b-bb64-b0ac9197a4af None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:03.365131 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c56b4-67bb-487c-9018-402dda537f68 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:03.365706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c56b4-67bb-487c-9018-402dda537f68 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:03.365951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c56b4-67bb-487c-9018-402dda537f68 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:03.367653 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 238/485] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:58:03.372648 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1794ef-1661-4472-8323-b76069890068 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:03.372941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1794ef-1661-4472-8323-b76069890068 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:03.373224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1794ef-1661-4472-8323-b76069890068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:03.403712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69693755-2e82-495b-bb64-b0ac9197a4af None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:03.407077 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-ac1794ef-1661-4472-8323-b76069890068 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:03.408282 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-ac1794ef-1661-4472-8323-b76069890068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:03.437868 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8f960-d568-4acf-bd14-5e39c5398c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:03.438683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8f960-d568-4acf-bd14-5e39c5398c1e None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:03.439004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8f960-d568-4acf-bd14-5e39c5398c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:03.439281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8f960-d568-4acf-bd14-5e39c5398c1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:03.450512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8f960-d568-4acf-bd14-5e39c5398c1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:03.451982 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8f960-d568-4acf-bd14-5e39c5398c1e None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:03.470376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-1ce8f960-d568-4acf-bd14-5e39c5398c1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:03.470376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:03.470376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:03.470376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:03.470376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:03.470376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:03.470376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:03.470376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:03.470376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:03.470376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:03.470376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:03.470376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:03.470376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:03.470376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:03.470376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:03.471276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:03.471276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:03.471276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:03.471276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:03.471276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:03.471276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:03.471276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:03.471276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:03.471276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:03.471276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:03.471276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:03.471276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:03.471276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:03.471276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:03.471276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:03.471276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:03.471276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:03.473135 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:03.473135 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:03.473135 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:03.473135 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:03.473135 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 248/486] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:03.480261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8f960-d568-4acf-bd14-5e39c5398c1e None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:03.563852 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 239/487] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:03.589803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8f960-d568-4acf-bd14-5e39c5398c1e None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:03.590471 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1794ef-1661-4472-8323-b76069890068 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:03.627327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65d1b6-662f-4eb4-b9c6-7416fda78a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:03.628194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65d1b6-662f-4eb4-b9c6-7416fda78a7b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:03.628565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65d1b6-662f-4eb4-b9c6-7416fda78a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:03.628848 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65d1b6-662f-4eb4-b9c6-7416fda78a7b None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:03.629574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65d1b6-662f-4eb4-b9c6-7416fda78a7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:03.629879 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65d1b6-662f-4eb4-b9c6-7416fda78a7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:03.631320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65d1b6-662f-4eb4-b9c6-7416fda78a7b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:03.650435 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 249/488] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:03.651620 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65d1b6-662f-4eb4-b9c6-7416fda78a7b None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:03.762429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65d1b6-662f-4eb4-b9c6-7416fda78a7b None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:03.801705 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0fb1da-a747-4ec4-a1c3-4795d0ec8b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:03.802481 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fb1da-a747-4ec4-a1c3-4795d0ec8b29 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:03.802764 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fb1da-a747-4ec4-a1c3-4795d0ec8b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:03.803091 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fb1da-a747-4ec4-a1c3-4795d0ec8b29 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:03.811040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0fb1da-a747-4ec4-a1c3-4795d0ec8b29 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:03.813232 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0fb1da-a747-4ec4-a1c3-4795d0ec8b29 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:03.831272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-6d0fb1da-a747-4ec4-a1c3-4795d0ec8b29 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 19:58:03.831272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:03.831272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:03.831272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:03.831272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:03.831272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:03.831272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:03.831272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:03.831272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:03.831272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:03.831272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:03.831272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:03.831272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:03.831272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:03.831272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:03.831272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:03.832771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:03.832771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:03.832771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:03.832771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:03.832771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:03.832771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:03.832771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:03.832771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:03.832771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:03.832771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:03.832771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:03.832771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:03.832771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:03.832771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:03.832771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:03.832771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:03.832771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:03.832771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:03.834119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:03.834119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:03.834119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:03.834119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:03.834119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:03.834119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:03.834119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:03.834119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:03.834119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 19:58:03.834119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:03.834119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 250/489] 10.222.0.65 () {56 vars in 1093 bytes} [Tue May 10 19:58:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:03.839398 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0fb1da-a747-4ec4-a1c3-4795d0ec8b29 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:03.872392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1794ef-1661-4472-8323-b76069890068 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:03.957333 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0fb1da-a747-4ec4-a1c3-4795d0ec8b29 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:03.958062 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e88ef-8642-4393-9e9c-a7d958dce095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:03.958890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e88ef-8642-4393-9e9c-a7d958dce095 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:03.959171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e88ef-8642-4393-9e9c-a7d958dce095 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:03.959446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e88ef-8642-4393-9e9c-a7d958dce095 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:03.969700 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e88ef-8642-4393-9e9c-a7d958dce095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:03.977434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-523e88ef-8642-4393-9e9c-a7d958dce095 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:03.977434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:03.977434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:03.977434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:03.977434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:03.977434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:03.977434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:03.977434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:03.977434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:03.977434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:03.977434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:03.977434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:03.977434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:03.977434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:03.977434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:03.977434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:03.978385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:03.978385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:03.978385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:03.978385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:03.978385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:03.978385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:03.978385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:03.978385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:03.978385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:03.978385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:03.978385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:03.978385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:03.978385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:03.978385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:03.978385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:03.978385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:03.979169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:03.979169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:03.979169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:03.979169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:03.979169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:03.979169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:03.979169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:03.979169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:03.979169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:03.979169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:03.979169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:03.979169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:03.987142 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 240/490] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:03.993315 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e88ef-8642-4393-9e9c-a7d958dce095 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:03.994167 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc1bad-3194-4108-852b-6d9774e396ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:03.994836 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc1bad-3194-4108-852b-6d9774e396ab None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:03.995113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc1bad-3194-4108-852b-6d9774e396ab None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:03.995388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc1bad-3194-4108-852b-6d9774e396ab None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:03.996233 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc1bad-3194-4108-852b-6d9774e396ab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:03.996538 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc1bad-3194-4108-852b-6d9774e396ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:03.997967 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc1bad-3194-4108-852b-6d9774e396ab None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:04.041740 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 251/491] 10.222.0.65 () {56 vars in 1133 bytes} [Tue May 10 19:58:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:04.048550 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fb7ac-daa0-4084-8f5f-427c76623ff4 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:04.048851 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fb7ac-daa0-4084-8f5f-427c76623ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:04.049112 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fb7ac-daa0-4084-8f5f-427c76623ff4 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:04.050741 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 252/492] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:04.051451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fb7ac-daa0-4084-8f5f-427c76623ff4 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:04.137836 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e88ef-8642-4393-9e9c-a7d958dce095 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:04.159992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fb7ac-daa0-4084-8f5f-427c76623ff4 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:04.172568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86199bde-0964-4721-9647-fd36f1b3f221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:04.173538 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86199bde-0964-4721-9647-fd36f1b3f221 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:04.174036 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86199bde-0964-4721-9647-fd36f1b3f221 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:04.174563 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86199bde-0964-4721-9647-fd36f1b3f221 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:04.175701 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86199bde-0964-4721-9647-fd36f1b3f221 None admin] RBAC: Adding query filter params (name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:04.176210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86199bde-0964-4721-9647-fd36f1b3f221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:04.177965 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86199bde-0964-4721-9647-fd36f1b3f221 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:04.186868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 241/493] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 19:58:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:04.188482 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a9d85-8713-4345-a6b1-7911b45dbb00 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:04.188984 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a9d85-8713-4345-a6b1-7911b45dbb00 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:04.189454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a9d85-8713-4345-a6b1-7911b45dbb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:04.194776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eea378-b6d5-426a-92fe-197efe39562c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:04.195420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eea378-b6d5-426a-92fe-197efe39562c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:04.195701 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eea378-b6d5-426a-92fe-197efe39562c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:04.195983 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eea378-b6d5-426a-92fe-197efe39562c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:04.210166 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-f92a9d85-8713-4345-a6b1-7911b45dbb00 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:04.212237 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f92a9d85-8713-4345-a6b1-7911b45dbb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:04.264625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eea378-b6d5-426a-92fe-197efe39562c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:04.266437 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eea378-b6d5-426a-92fe-197efe39562c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:04.281079 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 242/494] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:04.282356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a9d85-8713-4345-a6b1-7911b45dbb00 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:04.375426 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 253/495] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:58:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:58:04.375426 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336cf4a-83fe-446c-b523-d0b77678f440 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:04.375426 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336cf4a-83fe-446c-b523-d0b77678f440 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:04.375426 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336cf4a-83fe-446c-b523-d0b77678f440 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:04.375426 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 254/496] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:04.375426 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336cf4a-83fe-446c-b523-d0b77678f440 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:04.385799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a9d85-8713-4345-a6b1-7911b45dbb00 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:04.421315 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f67acb-b73e-4f85-a9ce-2254858b960f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:04.422383 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f67acb-b73e-4f85-a9ce-2254858b960f None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:04.422669 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f67acb-b73e-4f85-a9ce-2254858b960f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:04.422969 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f67acb-b73e-4f85-a9ce-2254858b960f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:04.434282 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f67acb-b73e-4f85-a9ce-2254858b960f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:04.435743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f67acb-b73e-4f85-a9ce-2254858b960f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:04.453971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-93f67acb-b73e-4f85-a9ce-2254858b960f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:04.453971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:04.453971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:04.453971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:04.453971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:04.453971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:04.453971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:04.453971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:04.453971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:04.453971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:04.453971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:04.453971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:04.453971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:04.453971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:04.453971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:04.454866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:04.454866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:04.454866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:04.454866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:04.454866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:04.454866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:04.454866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:04.454866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:04.454866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:04.454866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:04.454866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:04.454866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:04.454866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:04.454866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:04.454866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:04.454866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:04.454866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:04.455709 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:04.455709 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:04.455709 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:04.455709 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:04.455709 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 243/497] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:04.455982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f67acb-b73e-4f85-a9ce-2254858b960f None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:04.485219 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336cf4a-83fe-446c-b523-d0b77678f440 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:04.519721 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471f00a-9a1c-454e-8ccd-a601e716e316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:04.520507 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471f00a-9a1c-454e-8ccd-a601e716e316 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:04.520789 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471f00a-9a1c-454e-8ccd-a601e716e316 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:04.521061 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471f00a-9a1c-454e-8ccd-a601e716e316 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:04.531320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471f00a-9a1c-454e-8ccd-a601e716e316 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:04.539046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-b471f00a-9a1c-454e-8ccd-a601e716e316 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:04.539046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:04.539046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:04.539046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:04.539046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:04.539046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:04.539046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:04.539046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:04.539046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:04.539046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:04.539046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:04.539046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:04.539046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:04.539046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:04.539046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:04.539046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:04.539920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:04.539920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:04.539920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:04.539920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:04.539920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:04.539920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:04.539920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:04.539920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:04.539920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:04.539920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:04.539920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:04.539920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:04.539920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:04.539920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:04.539920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:04.539920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:04.540816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:04.540816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:04.540816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:04.540816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:04.540816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:04.540816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:04.540816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:04.540816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:04.540816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:04.540816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:04.540816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:04.540816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:04.540816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 255/498] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:04.541471 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8f920-e25a-4647-b9c2-76d3b0768526 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:04.542022 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8f920-e25a-4647-b9c2-76d3b0768526 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:04.542022 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8f920-e25a-4647-b9c2-76d3b0768526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:04.558799 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-22e8f920-e25a-4647-b9c2-76d3b0768526 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:04.559855 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f67acb-b73e-4f85-a9ce-2254858b960f None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:04.559970 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-22e8f920-e25a-4647-b9c2-76d3b0768526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:04.593556 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b86617-dd7c-41ab-a36b-9b44b2cef656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:04.594374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b86617-dd7c-41ab-a36b-9b44b2cef656 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:04.594664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b86617-dd7c-41ab-a36b-9b44b2cef656 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:04.594945 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b86617-dd7c-41ab-a36b-9b44b2cef656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:04.605758 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b86617-dd7c-41ab-a36b-9b44b2cef656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:04.607249 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b86617-dd7c-41ab-a36b-9b44b2cef656 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:04.625183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-17b86617-dd7c-41ab-a36b-9b44b2cef656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:04.625183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:04.625183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:04.625183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:04.625183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:04.625183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:04.625183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:04.625183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:04.625183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:04.625183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:04.625183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:04.625183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:04.625183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:04.625183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:04.625183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:04.626055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:04.626055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:04.626055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:04.626055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:04.626055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:04.626055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:04.626055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:04.626055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:04.626055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:04.626055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:04.626055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:04.626055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:04.626055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:04.626055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:04.626055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:04.626055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:04.626055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:04.626951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:04.626951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:04.626951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:04.626951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:04.627182 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 256/499] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:04.627706 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8f920-e25a-4647-b9c2-76d3b0768526 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:04.635658 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 244/500] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:04.635658 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b86617-dd7c-41ab-a36b-9b44b2cef656 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:04.734827 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8f920-e25a-4647-b9c2-76d3b0768526 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:04.739611 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b86617-dd7c-41ab-a36b-9b44b2cef656 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:04.769898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d4f0f-8337-41d9-ba7f-6def75a04a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:04.770593 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d4f0f-8337-41d9-ba7f-6def75a04a49 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:04.770881 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d4f0f-8337-41d9-ba7f-6def75a04a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:04.771155 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d4f0f-8337-41d9-ba7f-6def75a04a49 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:04.771889 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d4f0f-8337-41d9-ba7f-6def75a04a49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:04.772197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d4f0f-8337-41d9-ba7f-6def75a04a49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:04.773042 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9f604-151f-4c26-90d5-ea051aa2efdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:04.773562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d4f0f-8337-41d9-ba7f-6def75a04a49 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:04.773716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9f604-151f-4c26-90d5-ea051aa2efdb None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:04.773973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9f604-151f-4c26-90d5-ea051aa2efdb None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:04.774283 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9f604-151f-4c26-90d5-ea051aa2efdb None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:04.775032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9f604-151f-4c26-90d5-ea051aa2efdb None admin] RBAC: Adding query filter params (name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:04.775341 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9f604-151f-4c26-90d5-ea051aa2efdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:04.776712 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9f604-151f-4c26-90d5-ea051aa2efdb None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:04.785415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 257/501] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:04.785563 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 245/502] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 19:58:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:04.786290 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9f604-151f-4c26-90d5-ea051aa2efdb None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:04.787047 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d4f0f-8337-41d9-ba7f-6def75a04a49 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:04.891900 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9f604-151f-4c26-90d5-ea051aa2efdb None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:04.894124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d4f0f-8337-41d9-ba7f-6def75a04a49 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:04.924104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d536f6-ada5-4a66-a7b1-1898a19c03e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:04.924754 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d536f6-ada5-4a66-a7b1-1898a19c03e7 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:04.925635 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d536f6-ada5-4a66-a7b1-1898a19c03e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:04.925906 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d536f6-ada5-4a66-a7b1-1898a19c03e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:04.926648 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac560083-f72c-4e50-99cb-d05a39f9753b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:04.926757 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d536f6-ada5-4a66-a7b1-1898a19c03e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:04.926963 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d536f6-ada5-4a66-a7b1-1898a19c03e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:04.927429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac560083-f72c-4e50-99cb-d05a39f9753b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:04.927716 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac560083-f72c-4e50-99cb-d05a39f9753b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:04.928000 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac560083-f72c-4e50-99cb-d05a39f9753b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:04.928326 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d536f6-ada5-4a66-a7b1-1898a19c03e7 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:04.937670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac560083-f72c-4e50-99cb-d05a39f9753b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:04.939600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 246/503] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:04.940293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d536f6-ada5-4a66-a7b1-1898a19c03e7 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:04.945337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-ac560083-f72c-4e50-99cb-d05a39f9753b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:04.945337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:04.945337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:04.945337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:04.945337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:04.945337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:04.945337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:04.945337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:04.945337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:04.945337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:04.945337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:04.945337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:04.945337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:04.945337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:04.945337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:04.945337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:04.946254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:04.946254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:04.946254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:04.946254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:04.946254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:04.946254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:04.946254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:04.946254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:04.946254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:04.946254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:04.946254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:04.946254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:04.946254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:04.946254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:04.946254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:04.946254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:04.947105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:04.947105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:04.947105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:04.947105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:04.947105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:04.947105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:04.947105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:04.947105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:04.947105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:04.947105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:04.947105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:04.947105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:04.947105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 258/504] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:04.947761 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac560083-f72c-4e50-99cb-d05a39f9753b None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:05.033215 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d536f6-ada5-4a66-a7b1-1898a19c03e7 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:05.044867 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac560083-f72c-4e50-99cb-d05a39f9753b None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:05.065421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0d250-1bc0-4ee2-9348-2c88888f4c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:05.066218 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0d250-1bc0-4ee2-9348-2c88888f4c37 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:05.066498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0d250-1bc0-4ee2-9348-2c88888f4c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:05.066766 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0d250-1bc0-4ee2-9348-2c88888f4c37 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:05.073561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0d250-1bc0-4ee2-9348-2c88888f4c37 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:05.075028 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0d250-1bc0-4ee2-9348-2c88888f4c37 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:05.076106 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a89bc-af8e-4f89-8f70-b7a37ffc602b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:05.076882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a89bc-af8e-4f89-8f70-b7a37ffc602b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:05.077171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a89bc-af8e-4f89-8f70-b7a37ffc602b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:05.077457 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a89bc-af8e-4f89-8f70-b7a37ffc602b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:05.090150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a89bc-af8e-4f89-8f70-b7a37ffc602b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:05.091040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-43a0d250-1bc0-4ee2-9348-2c88888f4c37 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 10 19:58:05.091040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:05.091040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:05.091040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:05.091040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:05.091040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:05.091040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:05.091040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:05.091040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:05.091040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:05.091040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:05.091040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:05.091040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:05.091040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:05.091040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:05.091040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:05.091951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:05.091951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:05.091951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:05.091951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:05.091951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:05.091951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:05.091951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:05.091951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:05.091951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:05.091951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:05.091951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:05.091951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:05.091951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:05.091951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:05.091951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:05.091951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:05.091951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:05.091951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:05.092851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:05.092851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:05.092851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:05.092851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:05.092851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:05.092851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:05.092851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:05.092851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:05.092851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 10 19:58:05.092851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:05.093354 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a89bc-af8e-4f89-8f70-b7a37ffc602b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:05.093421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 247/505] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:58:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:05.093421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee90fea-0d5d-4095-ac94-9329be05371c None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:05.093770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee90fea-0d5d-4095-ac94-9329be05371c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:05.093999 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee90fea-0d5d-4095-ac94-9329be05371c None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:05.095220 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 248/506] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:05.095907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee90fea-0d5d-4095-ac94-9329be05371c None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:05.113671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-8e7a89bc-af8e-4f89-8f70-b7a37ffc602b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:05.113671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:05.113671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:05.113671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:05.113671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:05.113671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:05.113671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:05.113671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:05.113671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:05.113671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:05.113671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:05.113671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:05.113671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:05.113671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:05.113671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:05.114582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:05.114582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:05.114582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:05.114582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:05.114582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:05.114582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:05.114582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:05.114582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:05.114582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:05.114582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:05.114582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:05.114582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:05.114582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:05.114582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:05.114582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:05.114582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:05.114582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:05.115769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:05.115769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:05.115769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:05.115769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:05.115769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 259/507] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:05.116038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a89bc-af8e-4f89-8f70-b7a37ffc602b None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:05.193816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee90fea-0d5d-4095-ac94-9329be05371c None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:05.208323 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a89bc-af8e-4f89-8f70-b7a37ffc602b None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:05.226658 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98976e73-abd8-4af1-83f4-95d08fa373bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:05.227436 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98976e73-abd8-4af1-83f4-95d08fa373bf None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:05.227717 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98976e73-abd8-4af1-83f4-95d08fa373bf None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:05.227992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98976e73-abd8-4af1-83f4-95d08fa373bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:05.234487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98976e73-abd8-4af1-83f4-95d08fa373bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:05.235948 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98976e73-abd8-4af1-83f4-95d08fa373bf None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:05.239987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd44c9-e9fe-41f6-86cf-81ef44a15893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:05.240901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd44c9-e9fe-41f6-86cf-81ef44a15893 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:05.241185 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd44c9-e9fe-41f6-86cf-81ef44a15893 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:05.241462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd44c9-e9fe-41f6-86cf-81ef44a15893 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:05.242240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd44c9-e9fe-41f6-86cf-81ef44a15893 None admin] RBAC: Adding query filter params (name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:05.242550 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd44c9-e9fe-41f6-86cf-81ef44a15893 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:05.243901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd44c9-e9fe-41f6-86cf-81ef44a15893 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:05.250486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-98976e73-abd8-4af1-83f4-95d08fa373bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:05.250486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:05.250486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:05.250486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:05.250486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:05.250486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:05.250486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:05.250486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:05.250486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:05.250486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:05.250486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:05.250486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:05.250486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:05.250486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:05.250486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:05.253396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:05.253396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:05.253396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:05.253396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:05.253396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:05.253396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:05.253396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:05.253396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:05.253396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:05.253396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:05.253396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:05.253396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:05.253396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:05.253396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:05.253396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:05.253396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:05.253396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:05.254281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:05.254281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:05.254281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:05.254281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:05.254281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:05.254281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:05.254281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:05.254281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:05.254702 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 260/508] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 19:58:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:05.254772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 249/509] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:05.254836 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd44c9-e9fe-41f6-86cf-81ef44a15893 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:05.254900 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c692ac2-cb3e-4647-9a44-d2504db80057 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:05.254900 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c692ac2-cb3e-4647-9a44-d2504db80057 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:05.254900 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c692ac2-cb3e-4647-9a44-d2504db80057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:05.270045 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-2c692ac2-cb3e-4647-9a44-d2504db80057 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:05.271283 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-2c692ac2-cb3e-4647-9a44-d2504db80057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:05.334287 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 250/510] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:05.335048 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c692ac2-cb3e-4647-9a44-d2504db80057 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:05.361847 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd44c9-e9fe-41f6-86cf-81ef44a15893 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:05.394496 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581eb5d-725a-4ed4-866d-4c36499efc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:05.395166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581eb5d-725a-4ed4-866d-4c36499efc7b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:05.395451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581eb5d-725a-4ed4-866d-4c36499efc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:05.395725 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581eb5d-725a-4ed4-866d-4c36499efc7b None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:05.396561 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9581eb5d-725a-4ed4-866d-4c36499efc7b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:05.396866 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9581eb5d-725a-4ed4-866d-4c36499efc7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:05.398313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9581eb5d-725a-4ed4-866d-4c36499efc7b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:05.414571 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 261/511] 10.222.0.65 () {56 vars in 1137 bytes} [Tue May 10 19:58:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:05.415347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581eb5d-725a-4ed4-866d-4c36499efc7b None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:05.430895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c692ac2-cb3e-4647-9a44-d2504db80057 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:05.463454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf3610-c678-4b9a-b209-d8735156afa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:05.464102 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf3610-c678-4b9a-b209-d8735156afa5 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:05.464388 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf3610-c678-4b9a-b209-d8735156afa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:05.464653 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf3610-c678-4b9a-b209-d8735156afa5 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:05.465367 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf3610-c678-4b9a-b209-d8735156afa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:05.465665 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf3610-c678-4b9a-b209-d8735156afa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:05.467047 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf3610-c678-4b9a-b209-d8735156afa5 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:05.477602 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 251/512] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:05.478392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf3610-c678-4b9a-b209-d8735156afa5 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:05.510355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581eb5d-725a-4ed4-866d-4c36499efc7b None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:05.543433 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a7147-a472-42d2-b122-d32ff1838790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:05.544082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a7147-a472-42d2-b122-d32ff1838790 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:05.544370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a7147-a472-42d2-b122-d32ff1838790 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:05.544651 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a7147-a472-42d2-b122-d32ff1838790 None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:05.545443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a7147-a472-42d2-b122-d32ff1838790 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:05.545754 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a7147-a472-42d2-b122-d32ff1838790 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:05.547218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a7147-a472-42d2-b122-d32ff1838790 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:05.564735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 262/513] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:05.565535 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a7147-a472-42d2-b122-d32ff1838790 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:05.574345 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf3610-c678-4b9a-b209-d8735156afa5 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:05.606187 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292e8e0-ab43-490b-b769-4351311073b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:05.606910 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292e8e0-ab43-490b-b769-4351311073b3 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:05.607195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292e8e0-ab43-490b-b769-4351311073b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:05.607471 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292e8e0-ab43-490b-b769-4351311073b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:05.618320 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292e8e0-ab43-490b-b769-4351311073b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:05.619756 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292e8e0-ab43-490b-b769-4351311073b3 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:05.636128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-f292e8e0-ab43-490b-b769-4351311073b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:05.636128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:05.636128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:05.636128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:05.636128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:05.636128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:05.636128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:05.636128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:05.636128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:05.636128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:05.636128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:05.636128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:05.636128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:05.636128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:05.636128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:05.636974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:05.636974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:05.636974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:05.636974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:05.636974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:05.636974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:05.636974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:05.636974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:05.636974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:05.636974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:05.636974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:05.636974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:05.636974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:05.636974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:05.636974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:05.636974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:05.636974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:05.637801 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:05.637801 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:05.637801 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:05.637801 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:05.637801 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 252/514] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:05.638071 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292e8e0-ab43-490b-b769-4351311073b3 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:05.663713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a7147-a472-42d2-b122-d32ff1838790 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:05.695645 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21009c98-3fa3-4acf-9984-ac73532a455c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:05.696422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21009c98-3fa3-4acf-9984-ac73532a455c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:05.696706 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21009c98-3fa3-4acf-9984-ac73532a455c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:05.696983 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21009c98-3fa3-4acf-9984-ac73532a455c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:05.707063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21009c98-3fa3-4acf-9984-ac73532a455c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:05.708486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21009c98-3fa3-4acf-9984-ac73532a455c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:05.725580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-21009c98-3fa3-4acf-9984-ac73532a455c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:05.725580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:05.725580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:05.725580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:05.725580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:05.725580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:05.725580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:05.725580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:05.725580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:05.725580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:05.725580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:05.725580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:05.725580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:05.725580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:05.725580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:05.726612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:05.726612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:05.726612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:05.726612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:05.726612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:05.726612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:05.726612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:05.726612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:05.726612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:05.726612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:05.726612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:05.726612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:05.726612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:05.726612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:05.726612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:05.726612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:05.726612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:05.727759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:05.727759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:05.727759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:05.727759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:05.727759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 263/515] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:05.728140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21009c98-3fa3-4acf-9984-ac73532a455c None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:05.734930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292e8e0-ab43-490b-b769-4351311073b3 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:05.766214 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207aa9f-446e-4fa7-9098-8063d566b9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:05.766851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207aa9f-446e-4fa7-9098-8063d566b9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:05.767127 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207aa9f-446e-4fa7-9098-8063d566b9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:05.767395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207aa9f-446e-4fa7-9098-8063d566b9e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:05.819238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21009c98-3fa3-4acf-9984-ac73532a455c None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:05.829574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207aa9f-446e-4fa7-9098-8063d566b9e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:05.831030 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207aa9f-446e-4fa7-9098-8063d566b9e4 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:05.851835 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb8a89-bc66-4e3d-beb4-f44bf3ab1422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:05.852607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb8a89-bc66-4e3d-beb4-f44bf3ab1422 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:05.852889 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb8a89-bc66-4e3d-beb4-f44bf3ab1422 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:05.853167 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb8a89-bc66-4e3d-beb4-f44bf3ab1422 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:05.859997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb8a89-bc66-4e3d-beb4-f44bf3ab1422 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:05.861367 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb8a89-bc66-4e3d-beb4-f44bf3ab1422 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:05.875685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-52bb8a89-bc66-4e3d-beb4-f44bf3ab1422 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 10 19:58:05.875685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:05.875685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:05.875685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:05.875685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:05.875685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:05.875685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:05.875685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:05.875685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:05.875685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:05.875685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:05.875685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:05.875685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:05.875685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:05.875685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:05.875685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:05.876718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:05.876718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:05.876718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:05.876718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:05.876718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:05.876718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:05.876718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:05.876718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:05.876718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:05.876718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:05.876718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:05.876718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:05.876718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:05.876718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:05.876718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:05.876718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:05.876718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:05.876718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:05.877626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:05.877626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:05.877626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:05.877626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:05.877626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:05.877626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:05.877626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:05.877626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:05.877626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 10 19:58:05.877626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:05.878745 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 264/516] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:05.879373 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb8a89-bc66-4e3d-beb4-f44bf3ab1422 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:05.924080 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 253/517] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:58:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:58:05.924907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207aa9f-446e-4fa7-9098-8063d566b9e4 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:05.973608 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb8a89-bc66-4e3d-beb4-f44bf3ab1422 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:06.005087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb467653-0516-45b0-8c48-e03581752cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:06.005732 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb467653-0516-45b0-8c48-e03581752cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:06.006005 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb467653-0516-45b0-8c48-e03581752cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:06.006305 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb467653-0516-45b0-8c48-e03581752cd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:06.007146 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb467653-0516-45b0-8c48-e03581752cd9 None admin] RBAC: Adding query filter params (role.id=55528a93cfa64162b2f9574fb5cdde79, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=ae922b2da4814d10bed1fa345700ac04) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:06.007464 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb467653-0516-45b0-8c48-e03581752cd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:06.008893 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb467653-0516-45b0-8c48-e03581752cd9 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:06.014093 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 265/518] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:58:05 2022] GET /identity/v3/role_assignments?user.id=ae922b2da4814d10bed1fa345700ac04&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=55528a93cfa64162b2f9574fb5cdde79 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:06.014873 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb467653-0516-45b0-8c48-e03581752cd9 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:06.015797 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207aa9f-446e-4fa7-9098-8063d566b9e4 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:06.056680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a206494-caa2-4375-82f1-70d714ac58c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:06.057358 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a206494-caa2-4375-82f1-70d714ac58c4 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:06.057636 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a206494-caa2-4375-82f1-70d714ac58c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:06.057907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a206494-caa2-4375-82f1-70d714ac58c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:06.058672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a206494-caa2-4375-82f1-70d714ac58c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:06.059163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a206494-caa2-4375-82f1-70d714ac58c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:06.060574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a206494-caa2-4375-82f1-70d714ac58c4 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:06.077023 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 254/519] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:06.077023 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a206494-caa2-4375-82f1-70d714ac58c4 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:06.129569 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb467653-0516-45b0-8c48-e03581752cd9 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:06.168309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0584ec-c029-4284-8385-d74f3db5989b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:06.168980 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0584ec-c029-4284-8385-d74f3db5989b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:06.169261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0584ec-c029-4284-8385-d74f3db5989b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:06.169537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0584ec-c029-4284-8385-d74f3db5989b None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:06.170318 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0584ec-c029-4284-8385-d74f3db5989b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:06.170631 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0584ec-c029-4284-8385-d74f3db5989b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:06.172048 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0584ec-c029-4284-8385-d74f3db5989b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:06.178577 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a206494-caa2-4375-82f1-70d714ac58c4 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:06.185944 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 266/520] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:06.186869 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0584ec-c029-4284-8385-d74f3db5989b None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:06.218471 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c5c25-2161-4f6f-b3bb-ad6d40806324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:06.219143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c5c25-2161-4f6f-b3bb-ad6d40806324 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:06.219427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c5c25-2161-4f6f-b3bb-ad6d40806324 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:06.219705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c5c25-2161-4f6f-b3bb-ad6d40806324 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:06.220540 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c5c25-2161-4f6f-b3bb-ad6d40806324 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:06.220847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c5c25-2161-4f6f-b3bb-ad6d40806324 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:06.222426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c5c25-2161-4f6f-b3bb-ad6d40806324 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:06.260376 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 255/521] 10.222.0.65 () {56 vars in 1139 bytes} [Tue May 10 19:58:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:06.261553 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c5c25-2161-4f6f-b3bb-ad6d40806324 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:06.288529 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0584ec-c029-4284-8385-d74f3db5989b None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:06.321981 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5750291-35f4-41be-a74f-a374ae2036c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:06.322800 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5750291-35f4-41be-a74f-a374ae2036c6 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:06.323094 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5750291-35f4-41be-a74f-a374ae2036c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:06.323353 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5750291-35f4-41be-a74f-a374ae2036c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:06.333917 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5750291-35f4-41be-a74f-a374ae2036c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:06.335388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5750291-35f4-41be-a74f-a374ae2036c6 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:06.352601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-a5750291-35f4-41be-a74f-a374ae2036c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:06.352601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:06.352601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:06.352601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:06.352601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:06.352601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:06.352601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:06.352601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:06.352601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:06.352601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:06.352601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:06.352601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:06.352601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:06.352601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:06.352601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:06.353685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:06.353685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:06.353685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:06.353685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:06.353685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:06.353685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:06.353685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:06.353685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:06.353685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:06.353685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:06.353685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:06.353685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:06.353685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:06.353685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:06.353685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:06.353685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:06.353685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:06.359234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:06.359234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:06.359234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:06.359234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:06.359234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 267/522] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:06.359234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5750291-35f4-41be-a74f-a374ae2036c6 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:06.369463 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c5c25-2161-4f6f-b3bb-ad6d40806324 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:06.404058 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e5591-ac46-4ab6-808b-ee79f185e7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:06.404884 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e5591-ac46-4ab6-808b-ee79f185e7ce None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:06.405174 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e5591-ac46-4ab6-808b-ee79f185e7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:06.405451 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e5591-ac46-4ab6-808b-ee79f185e7ce None admin] PATH_INFO: `/v3/users/glance` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:06.412720 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e5591-ac46-4ab6-808b-ee79f185e7ce None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:06.414190 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e5591-ac46-4ab6-808b-ee79f185e7ce None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:06.429597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-c70e5591-ac46-4ab6-808b-ee79f185e7ce None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 10 19:58:06.429597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:06.429597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:06.429597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:06.429597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:06.429597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:06.429597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:06.429597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:06.429597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:06.429597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:06.429597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:06.429597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:06.429597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:06.429597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:06.429597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:06.429597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:06.430462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:06.430462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:06.430462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:06.430462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:06.430462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:06.430462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:06.430462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:06.430462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:06.430462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:06.430462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:06.430462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:06.430462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:06.430462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:06.430462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:06.430462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:06.430462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:06.430462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:06.430462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:06.431373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:06.431373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:06.431373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:06.431373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:06.431373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:06.431373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:06.431373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:06.431373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:06.431373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 10 19:58:06.431373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:06.435490 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 256/523] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:58:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:06.436245 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e5591-ac46-4ab6-808b-ee79f185e7ce None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:06.452893 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5750291-35f4-41be-a74f-a374ae2036c6 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:06.486005 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46045156-0bf1-4a46-8a2b-a23e13f264ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:06.486704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46045156-0bf1-4a46-8a2b-a23e13f264ef None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:06.486993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46045156-0bf1-4a46-8a2b-a23e13f264ef None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:06.487272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46045156-0bf1-4a46-8a2b-a23e13f264ef None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:06.488527 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46045156-0bf1-4a46-8a2b-a23e13f264ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:06.489951 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46045156-0bf1-4a46-8a2b-a23e13f264ef None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:06.533371 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e5591-ac46-4ab6-808b-ee79f185e7ce None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:06.566913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720e5a1-e1ed-471f-b8c2-5d894b05eea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:06.567670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720e5a1-e1ed-471f-b8c2-5d894b05eea2 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:06.567955 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720e5a1-e1ed-471f-b8c2-5d894b05eea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:06.568228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720e5a1-e1ed-471f-b8c2-5d894b05eea2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:06.602889 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 268/524] 10.222.0.65 () {60 vars in 1132 bytes} [Tue May 10 19:58:06 2022] POST /identity/v3/users => generated 247 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:58:06.615547 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46045156-0bf1-4a46-8a2b-a23e13f264ef None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:06.638245 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720e5a1-e1ed-471f-b8c2-5d894b05eea2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:06.640046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720e5a1-e1ed-471f-b8c2-5d894b05eea2 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:06.730870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46045156-0bf1-4a46-8a2b-a23e13f264ef None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:06.748675 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 257/525] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:58:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:58:06.749378 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720e5a1-e1ed-471f-b8c2-5d894b05eea2 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:06.773937 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03958194-9282-4f62-8435-f5e64d3557d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:06.774859 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03958194-9282-4f62-8435-f5e64d3557d2 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:06.775243 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03958194-9282-4f62-8435-f5e64d3557d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:06.775639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03958194-9282-4f62-8435-f5e64d3557d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:06.776556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03958194-9282-4f62-8435-f5e64d3557d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:06.776970 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03958194-9282-4f62-8435-f5e64d3557d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:06.778537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03958194-9282-4f62-8435-f5e64d3557d2 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:06.791671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 269/526] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:06.792741 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03958194-9282-4f62-8435-f5e64d3557d2 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:06.853847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720e5a1-e1ed-471f-b8c2-5d894b05eea2 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:06.889922 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f45d4f-c266-4faa-bf76-ba1409451295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:06.890879 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f45d4f-c266-4faa-bf76-ba1409451295 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:06.891311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f45d4f-c266-4faa-bf76-ba1409451295 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:06.891731 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f45d4f-c266-4faa-bf76-ba1409451295 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:06.892728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f45d4f-c266-4faa-bf76-ba1409451295 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:06.893180 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f45d4f-c266-4faa-bf76-ba1409451295 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:06.895763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f45d4f-c266-4faa-bf76-ba1409451295 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:06.900974 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03958194-9282-4f62-8435-f5e64d3557d2 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:06.927694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 258/527] 10.222.0.65 () {56 vars in 1137 bytes} [Tue May 10 19:58:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:06.929214 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f45d4f-c266-4faa-bf76-ba1409451295 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:06.937032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b321ef5-509f-4956-a90f-ac5346a840f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:06.937821 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b321ef5-509f-4956-a90f-ac5346a840f1 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:06.938134 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b321ef5-509f-4956-a90f-ac5346a840f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:06.938681 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b321ef5-509f-4956-a90f-ac5346a840f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:06.948634 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b321ef5-509f-4956-a90f-ac5346a840f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:06.950105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b321ef5-509f-4956-a90f-ac5346a840f1 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:06.967486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-4b321ef5-509f-4956-a90f-ac5346a840f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:06.967486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:06.967486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:06.967486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:06.967486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:06.967486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:06.967486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:06.967486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:06.967486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:06.967486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:06.967486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:06.967486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:06.967486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:06.967486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:06.967486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:06.968369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:06.968369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:06.968369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:06.968369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:06.968369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:06.968369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:06.968369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:06.968369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:06.968369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:06.968369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:06.968369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:06.968369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:06.968369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:06.968369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:06.968369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:06.968369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:06.968369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:06.969218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:06.969218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:06.969218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:06.969218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:06.973703 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 270/528] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:06.974600 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b321ef5-509f-4956-a90f-ac5346a840f1 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:07.036824 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f45d4f-c266-4faa-bf76-ba1409451295 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:07.071926 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5c747-bf7b-4c22-b2a5-d7d37f846787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:07.073099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5c747-bf7b-4c22-b2a5-d7d37f846787 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:07.073588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5c747-bf7b-4c22-b2a5-d7d37f846787 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:07.074124 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5c747-bf7b-4c22-b2a5-d7d37f846787 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:07.074299 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b321ef5-509f-4956-a90f-ac5346a840f1 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:07.083699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5c747-bf7b-4c22-b2a5-d7d37f846787 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:07.086398 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5c747-bf7b-4c22-b2a5-d7d37f846787 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:07.101289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-6ae5c747-bf7b-4c22-b2a5-d7d37f846787 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:07.101289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:07.101289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:07.101289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:07.101289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:07.101289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:07.101289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:07.101289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:07.101289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:07.101289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:07.101289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:07.101289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:07.101289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:07.101289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:07.101289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:07.102769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:07.102769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:07.102769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:07.102769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:07.102769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:07.102769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:07.102769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:07.102769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:07.102769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:07.102769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:07.102769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:07.102769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:07.102769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:07.102769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:07.102769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:07.102769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:07.102769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:07.104808 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:07.104808 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:07.104808 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:07.104808 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:07.104808 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:07.104808 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:07.104808 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:07.104808 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:07.111756 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef3fbf-e0cf-4a3f-adc5-e83a983b41cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:07.111756 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef3fbf-e0cf-4a3f-adc5-e83a983b41cb None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:07.111756 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef3fbf-e0cf-4a3f-adc5-e83a983b41cb None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:07.111756 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef3fbf-e0cf-4a3f-adc5-e83a983b41cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:07.112302 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 259/529] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:07.113937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5c747-bf7b-4c22-b2a5-d7d37f846787 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:07.179886 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef3fbf-e0cf-4a3f-adc5-e83a983b41cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:07.181360 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef3fbf-e0cf-4a3f-adc5-e83a983b41cb None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:07.220535 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5c747-bf7b-4c22-b2a5-d7d37f846787 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:07.256416 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757cc53-fa8f-4020-bb33-f4b7ce2399dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:07.257412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757cc53-fa8f-4020-bb33-f4b7ce2399dc None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:07.257863 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757cc53-fa8f-4020-bb33-f4b7ce2399dc None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:07.258350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757cc53-fa8f-4020-bb33-f4b7ce2399dc None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:07.264259 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6757cc53-fa8f-4020-bb33-f4b7ce2399dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:07.264259 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6757cc53-fa8f-4020-bb33-f4b7ce2399dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:07.264259 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6757cc53-fa8f-4020-bb33-f4b7ce2399dc None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:07.281176 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 271/530] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:58:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:58:07.282127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef3fbf-e0cf-4a3f-adc5-e83a983b41cb None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:07.285219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 260/531] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:07.286616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757cc53-fa8f-4020-bb33-f4b7ce2399dc None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:07.384233 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef3fbf-e0cf-4a3f-adc5-e83a983b41cb None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:07.391795 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757cc53-fa8f-4020-bb33-f4b7ce2399dc None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:07.418497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05838c0-856f-46a6-9f63-87e253dda65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:07.419158 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05838c0-856f-46a6-9f63-87e253dda65c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:07.419442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05838c0-856f-46a6-9f63-87e253dda65c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:07.419720 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05838c0-856f-46a6-9f63-87e253dda65c None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:07.420517 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05838c0-856f-46a6-9f63-87e253dda65c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:07.420830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05838c0-856f-46a6-9f63-87e253dda65c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:07.422290 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05838c0-856f-46a6-9f63-87e253dda65c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:07.434015 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 272/532] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:07.434939 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05838c0-856f-46a6-9f63-87e253dda65c None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:07.438659 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa3408-82ff-4fe2-8831-69fc8fffcad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:07.439645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa3408-82ff-4fe2-8831-69fc8fffcad5 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:07.440090 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa3408-82ff-4fe2-8831-69fc8fffcad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:07.440541 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa3408-82ff-4fe2-8831-69fc8fffcad5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:07.451727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa3408-82ff-4fe2-8831-69fc8fffcad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:07.453444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa3408-82ff-4fe2-8831-69fc8fffcad5 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:07.472865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-81aa3408-82ff-4fe2-8831-69fc8fffcad5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:07.472865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:07.472865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:07.472865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:07.472865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:07.472865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:07.472865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:07.472865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:07.472865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:07.472865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:07.472865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:07.472865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:07.472865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:07.472865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:07.472865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:07.474029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:07.474029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:07.474029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:07.474029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:07.474029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:07.474029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:07.474029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:07.474029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:07.474029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:07.474029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:07.474029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:07.474029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:07.474029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:07.474029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:07.474029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:07.474029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:07.474029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:07.475036 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:07.475036 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:07.475036 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:07.475036 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:07.476826 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 261/533] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:07.477916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa3408-82ff-4fe2-8831-69fc8fffcad5 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:07.534954 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05838c0-856f-46a6-9f63-87e253dda65c None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:07.568204 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9cdab7-2e37-443d-83ce-85c05dc3e6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:07.568983 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cdab7-2e37-443d-83ce-85c05dc3e6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:07.569268 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cdab7-2e37-443d-83ce-85c05dc3e6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:07.569539 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cdab7-2e37-443d-83ce-85c05dc3e6a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:07.576346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9cdab7-2e37-443d-83ce-85c05dc3e6a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:07.577793 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9cdab7-2e37-443d-83ce-85c05dc3e6a6 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:07.579843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa3408-82ff-4fe2-8831-69fc8fffcad5 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:07.593203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-6a9cdab7-2e37-443d-83ce-85c05dc3e6a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:07.593203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:07.593203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:07.593203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:07.593203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:07.593203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:07.593203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:07.593203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:07.593203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:07.593203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:07.593203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:07.593203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:07.593203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:07.593203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:07.593203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:07.594331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:07.594331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:07.594331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:07.594331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:07.594331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:07.594331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:07.594331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:07.594331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:07.594331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:07.594331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:07.594331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:07.594331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:07.594331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:07.594331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:07.594331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:07.594331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:07.594331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:07.595382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:07.595382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:07.595382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:07.595382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:07.595382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:07.595382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:07.595382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:07.595382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:07.598369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 273/534] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:07.599018 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9cdab7-2e37-443d-83ce-85c05dc3e6a6 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:07.619755 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba187c-1843-4493-a9ff-b7507948d301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:07.620996 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba187c-1843-4493-a9ff-b7507948d301 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:07.621760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba187c-1843-4493-a9ff-b7507948d301 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:07.622498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba187c-1843-4493-a9ff-b7507948d301 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:07.624409 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba187c-1843-4493-a9ff-b7507948d301 None admin] RBAC: Adding query filter params (role.id=55528a93cfa64162b2f9574fb5cdde79, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=0e366ec240e5464d8c5e92f807123c23) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:07.625253 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba187c-1843-4493-a9ff-b7507948d301 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:07.628984 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba187c-1843-4493-a9ff-b7507948d301 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:07.636802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 262/535] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:58:07 2022] GET /identity/v3/role_assignments?user.id=0e366ec240e5464d8c5e92f807123c23&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=55528a93cfa64162b2f9574fb5cdde79 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:07.637880 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba187c-1843-4493-a9ff-b7507948d301 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:07.704740 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9cdab7-2e37-443d-83ce-85c05dc3e6a6 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:07.740429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e3178-9479-4a1f-81cf-6233a9a129fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:07.741125 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e3178-9479-4a1f-81cf-6233a9a129fd None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:07.741402 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e3178-9479-4a1f-81cf-6233a9a129fd None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:07.741749 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e3178-9479-4a1f-81cf-6233a9a129fd None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:07.742595 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e3178-9479-4a1f-81cf-6233a9a129fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:07.742905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e3178-9479-4a1f-81cf-6233a9a129fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:07.744572 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e3178-9479-4a1f-81cf-6233a9a129fd None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:07.747812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba187c-1843-4493-a9ff-b7507948d301 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:07.757522 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 274/536] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:07.762304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e3178-9479-4a1f-81cf-6233a9a129fd None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:07.785633 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499da7b-10fe-4535-9425-5faabe65830a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:07.786649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499da7b-10fe-4535-9425-5faabe65830a None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:07.787167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499da7b-10fe-4535-9425-5faabe65830a None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:07.787672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499da7b-10fe-4535-9425-5faabe65830a None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:07.788672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2499da7b-10fe-4535-9425-5faabe65830a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:07.789276 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2499da7b-10fe-4535-9425-5faabe65830a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:07.790992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2499da7b-10fe-4535-9425-5faabe65830a None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:07.809607 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 263/537] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:07.831953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499da7b-10fe-4535-9425-5faabe65830a None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:07.888273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e3178-9479-4a1f-81cf-6233a9a129fd None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:07.931540 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be3378-5ef0-47aa-9647-5b5628b014db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:07.932367 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be3378-5ef0-47aa-9647-5b5628b014db None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:07.932630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be3378-5ef0-47aa-9647-5b5628b014db None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:07.932907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be3378-5ef0-47aa-9647-5b5628b014db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:07.942593 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be3378-5ef0-47aa-9647-5b5628b014db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:07.944215 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be3378-5ef0-47aa-9647-5b5628b014db None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:07.963443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-11be3378-5ef0-47aa-9647-5b5628b014db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:07.963443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:07.963443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:07.963443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:07.963443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:07.963443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:07.963443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:07.963443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:07.963443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:07.963443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:07.963443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:07.963443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:07.963443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:07.963443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:07.963443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:07.964382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:07.964382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:07.964382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:07.964382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:07.964382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:07.964382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:07.964382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:07.964382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:07.964382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:07.964382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:07.964382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:07.964382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:07.964382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:07.964382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:07.964382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:07.964382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:07.964382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:07.965237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:07.965237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:07.965237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:07.965237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:07.965237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:07.965237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:07.965237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:07.965237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:07.965237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 275/538] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:07.969993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be3378-5ef0-47aa-9647-5b5628b014db None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:08.012291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499da7b-10fe-4535-9425-5faabe65830a None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:08.082485 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be3378-5ef0-47aa-9647-5b5628b014db None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:08.103712 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d628dfb-40b5-4921-a3bb-b809c4dce308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:08.104380 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d628dfb-40b5-4921-a3bb-b809c4dce308 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:08.104661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d628dfb-40b5-4921-a3bb-b809c4dce308 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:08.104936 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d628dfb-40b5-4921-a3bb-b809c4dce308 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:08.105780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d628dfb-40b5-4921-a3bb-b809c4dce308 None admin] RBAC: Adding query filter params (role.id=55528a93cfa64162b2f9574fb5cdde79, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=a6aabd1571d04bcfae842626fef5ed6d) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:08.106105 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d628dfb-40b5-4921-a3bb-b809c4dce308 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:08.107561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d628dfb-40b5-4921-a3bb-b809c4dce308 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:08.119670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ba8a5-fe2b-41d3-b526-fc312cc23566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:08.120626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ba8a5-fe2b-41d3-b526-fc312cc23566 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:08.121014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ba8a5-fe2b-41d3-b526-fc312cc23566 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:08.121393 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ba8a5-fe2b-41d3-b526-fc312cc23566 None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:08.122339 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ba8a5-fe2b-41d3-b526-fc312cc23566 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:08.122751 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ba8a5-fe2b-41d3-b526-fc312cc23566 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:08.124299 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ba8a5-fe2b-41d3-b526-fc312cc23566 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:08.128937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 264/539] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:58:07 2022] GET /identity/v3/role_assignments?user.id=a6aabd1571d04bcfae842626fef5ed6d&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=55528a93cfa64162b2f9574fb5cdde79 => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:08.147947 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 276/540] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:08.149213 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d628dfb-40b5-4921-a3bb-b809c4dce308 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:08.293565 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d628dfb-40b5-4921-a3bb-b809c4dce308 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:08.328561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a915aa-b20b-43bb-9ee3-0a3e6d14ba91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:08.329224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a915aa-b20b-43bb-9ee3-0a3e6d14ba91 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:08.329559 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a915aa-b20b-43bb-9ee3-0a3e6d14ba91 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:08.329841 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a915aa-b20b-43bb-9ee3-0a3e6d14ba91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:08.330720 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a915aa-b20b-43bb-9ee3-0a3e6d14ba91 None admin] RBAC: Adding query filter params (role.id=55528a93cfa64162b2f9574fb5cdde79, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=67a252cde71a455fb1dc4bf9c47c0490) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:08.331055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a915aa-b20b-43bb-9ee3-0a3e6d14ba91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:08.332508 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a915aa-b20b-43bb-9ee3-0a3e6d14ba91 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:08.338855 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 265/541] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:58:08 2022] GET /identity/v3/role_assignments?user.id=67a252cde71a455fb1dc4bf9c47c0490&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=55528a93cfa64162b2f9574fb5cdde79 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:08.397941 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a5b61-7270-47a3-9d6b-61f09a543014 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:08.398349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a5b61-7270-47a3-9d6b-61f09a543014 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:08.398571 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a5b61-7270-47a3-9d6b-61f09a543014 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:08.399948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 277/542] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:08.405611 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b2002-99af-41f5-8bbc-c5166adc5ecf None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:08.405902 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b2002-99af-41f5-8bbc-c5166adc5ecf None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:08.406210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b2002-99af-41f5-8bbc-c5166adc5ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:08.434766 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-822b2002-99af-41f5-8bbc-c5166adc5ecf None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:08.435991 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-822b2002-99af-41f5-8bbc-c5166adc5ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:08.523233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 266/543] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:08.552100 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a5b61-7270-47a3-9d6b-61f09a543014 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:08.676468 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a5b61-7270-47a3-9d6b-61f09a543014 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:08.724399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225ca88-f7da-4af2-ae5c-97f4d981c951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:08.725392 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225ca88-f7da-4af2-ae5c-97f4d981c951 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:08.725843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225ca88-f7da-4af2-ae5c-97f4d981c951 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:08.726262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225ca88-f7da-4af2-ae5c-97f4d981c951 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:08.739489 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225ca88-f7da-4af2-ae5c-97f4d981c951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:08.747816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-d225ca88-f7da-4af2-ae5c-97f4d981c951 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:08.747816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:08.747816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:08.747816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:08.747816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:08.747816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:08.747816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:08.747816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:08.747816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:08.747816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:08.747816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:08.747816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:08.747816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:08.747816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:08.747816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:08.747816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:08.748905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:08.748905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:08.748905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:08.748905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:08.748905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:08.748905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:08.748905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:08.748905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:08.748905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:08.748905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:08.748905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:08.748905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:08.748905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:08.748905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:08.748905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:08.748905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:08.749915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:08.749915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:08.749915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:08.749915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:08.749915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:08.749915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:08.749915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:08.749915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:08.749915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:08.749915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:08.749915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:08.749915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:08.754106 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 278/544] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:08.757729 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b2002-99af-41f5-8bbc-c5166adc5ecf None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:08.921459 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b2002-99af-41f5-8bbc-c5166adc5ecf None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:08.958285 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189834b-0f90-4383-86a5-ef0a6db42794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:08.959381 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189834b-0f90-4383-86a5-ef0a6db42794 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:08.959894 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189834b-0f90-4383-86a5-ef0a6db42794 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:08.960387 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189834b-0f90-4383-86a5-ef0a6db42794 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:08.961343 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189834b-0f90-4383-86a5-ef0a6db42794 None admin] RBAC: Adding query filter params (name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:08.961865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189834b-0f90-4383-86a5-ef0a6db42794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:08.963459 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189834b-0f90-4383-86a5-ef0a6db42794 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:08.972642 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 267/545] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 19:58:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:08.977782 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225ca88-f7da-4af2-ae5c-97f4d981c951 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:09.130347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225ca88-f7da-4af2-ae5c-97f4d981c951 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:09.179686 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b3b9d-c98b-412f-8bfd-cb122f778266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:09.180486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b3b9d-c98b-412f-8bfd-cb122f778266 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:09.180768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b3b9d-c98b-412f-8bfd-cb122f778266 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:09.181045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b3b9d-c98b-412f-8bfd-cb122f778266 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:09.198233 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b3b9d-c98b-412f-8bfd-cb122f778266 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:09.199855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b3b9d-c98b-412f-8bfd-cb122f778266 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:09.221064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-dc2b3b9d-c98b-412f-8bfd-cb122f778266 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:09.221064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:09.221064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:09.221064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:09.221064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:09.221064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:09.221064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:09.221064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:09.221064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:09.221064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:09.221064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:09.221064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:09.221064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:09.221064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:09.221064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:09.222075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:09.222075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:09.222075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:09.222075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:09.222075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:09.222075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:09.222075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:09.222075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:09.222075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:09.222075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:09.222075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:09.222075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:09.222075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:09.222075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:09.222075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:09.222075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:09.222075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:09.222970 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:09.222970 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:09.222970 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:09.222970 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:09.222970 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 279/546] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:09.230500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189834b-0f90-4383-86a5-ef0a6db42794 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:09.292268 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d0edf-421a-4818-a6b5-195d6e6b9ccd None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:09.292580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d0edf-421a-4818-a6b5-195d6e6b9ccd None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:09.292822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d0edf-421a-4818-a6b5-195d6e6b9ccd None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:09.294488 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 280/547] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:58:09.300519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76b4f5-8b35-4cc6-9ea5-39e6502b6981 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:09.300806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76b4f5-8b35-4cc6-9ea5-39e6502b6981 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:09.301088 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76b4f5-8b35-4cc6-9ea5-39e6502b6981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:09.320155 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-6f76b4f5-8b35-4cc6-9ea5-39e6502b6981 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:09.321753 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-6f76b4f5-8b35-4cc6-9ea5-39e6502b6981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:09.452294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189834b-0f90-4383-86a5-ef0a6db42794 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:09.465238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 281/548] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:09.501578 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653b19a-5f0a-460f-9ae7-653d228244fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:09.502208 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653b19a-5f0a-460f-9ae7-653d228244fe None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:09.502496 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653b19a-5f0a-460f-9ae7-653d228244fe None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:09.502773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653b19a-5f0a-460f-9ae7-653d228244fe None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:09.504061 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653b19a-5f0a-460f-9ae7-653d228244fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:09.504368 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653b19a-5f0a-460f-9ae7-653d228244fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:09.505736 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653b19a-5f0a-460f-9ae7-653d228244fe None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:09.526776 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 268/549] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:09.528347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76b4f5-8b35-4cc6-9ea5-39e6502b6981 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:09.528807 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653b19a-5f0a-460f-9ae7-653d228244fe None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:09.638675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76b4f5-8b35-4cc6-9ea5-39e6502b6981 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:09.672644 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350559e-9639-4f41-baf5-f00f193e2386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:09.673434 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350559e-9639-4f41-baf5-f00f193e2386 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:09.673716 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350559e-9639-4f41-baf5-f00f193e2386 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:09.673993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350559e-9639-4f41-baf5-f00f193e2386 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:09.691192 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350559e-9639-4f41-baf5-f00f193e2386 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:09.700356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-f350559e-9639-4f41-baf5-f00f193e2386 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:09.700356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:09.700356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:09.700356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:09.700356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:09.700356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:09.700356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:09.700356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:09.700356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:09.700356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:09.700356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:09.700356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:09.700356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:09.700356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:09.700356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:09.700356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:09.701292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:09.701292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:09.701292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:09.701292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:09.701292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:09.701292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:09.701292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:09.701292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:09.701292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:09.701292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:09.701292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:09.701292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:09.701292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:09.701292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:09.701292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:09.701292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:09.702113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:09.702113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:09.702113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:09.702113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:09.702113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:09.702113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:09.702113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:09.702113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:09.702113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:09.702113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:09.702113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:09.702113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:09.702113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 282/550] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:09.709494 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350559e-9639-4f41-baf5-f00f193e2386 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:09.728868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653b19a-5f0a-460f-9ae7-653d228244fe None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:09.803419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7dcf1-de61-4cb2-938e-8a7a9679a73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:09.804141 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7dcf1-de61-4cb2-938e-8a7a9679a73b None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:09.804422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7dcf1-de61-4cb2-938e-8a7a9679a73b None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:09.804692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7dcf1-de61-4cb2-938e-8a7a9679a73b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:09.815524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7dcf1-de61-4cb2-938e-8a7a9679a73b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:09.816981 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7dcf1-de61-4cb2-938e-8a7a9679a73b None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:09.841335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-d1a7dcf1-de61-4cb2-938e-8a7a9679a73b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 19:58:09.841335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:09.841335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:09.841335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:09.841335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:09.841335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:09.841335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:09.841335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:09.841335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:09.841335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:09.841335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:09.841335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:09.841335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:09.841335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:09.841335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:09.841335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:09.842303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:09.842303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:09.842303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:09.842303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:09.842303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:09.842303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:09.842303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:09.842303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:09.842303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:09.842303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:09.842303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:09.842303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:09.842303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:09.842303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:09.842303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:09.842303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:09.842303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:09.842303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:09.843206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:09.843206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:09.843206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:09.843206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:09.843206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:09.843206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:09.843206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:09.843206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:09.843206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 19:58:09.843206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:09.843206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 269/551] 10.222.0.65 () {56 vars in 1093 bytes} [Tue May 10 19:58:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:09.845627 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350559e-9639-4f41-baf5-f00f193e2386 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:09.851014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7dcf1-de61-4cb2-938e-8a7a9679a73b None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:09.879274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b938468-334e-47bc-b116-93301301357f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:09.879937 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b938468-334e-47bc-b116-93301301357f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:09.880221 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b938468-334e-47bc-b116-93301301357f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:09.880494 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b938468-334e-47bc-b116-93301301357f None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:09.881242 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b938468-334e-47bc-b116-93301301357f None admin] RBAC: Adding query filter params (name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:09.881546 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b938468-334e-47bc-b116-93301301357f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:09.882931 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b938468-334e-47bc-b116-93301301357f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:09.891369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 283/552] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 19:58:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:09.899015 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b938468-334e-47bc-b116-93301301357f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:10.037926 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7dcf1-de61-4cb2-938e-8a7a9679a73b None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:10.095799 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b938468-334e-47bc-b116-93301301357f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:10.105774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52ad09-ec77-4346-92ea-91956f9d4016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:10.106478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52ad09-ec77-4346-92ea-91956f9d4016 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:10.106760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52ad09-ec77-4346-92ea-91956f9d4016 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:10.107033 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52ad09-ec77-4346-92ea-91956f9d4016 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:10.107884 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52ad09-ec77-4346-92ea-91956f9d4016 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:10.108192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52ad09-ec77-4346-92ea-91956f9d4016 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:10.111328 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52ad09-ec77-4346-92ea-91956f9d4016 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:10.141785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 270/553] 10.222.0.65 () {56 vars in 1133 bytes} [Tue May 10 19:58:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:10.147888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52ad09-ec77-4346-92ea-91956f9d4016 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:10.176854 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3016f-c567-427a-bf51-9a593b0579f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:10.177667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3016f-c567-427a-bf51-9a593b0579f4 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:10.177956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3016f-c567-427a-bf51-9a593b0579f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:10.178241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3016f-c567-427a-bf51-9a593b0579f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:10.195983 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3016f-c567-427a-bf51-9a593b0579f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:10.197436 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3016f-c567-427a-bf51-9a593b0579f4 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:10.228045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-dac3016f-c567-427a-bf51-9a593b0579f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:10.228045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:10.228045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:10.228045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:10.228045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:10.228045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:10.228045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:10.228045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:10.228045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:10.228045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:10.228045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:10.228045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:10.228045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:10.228045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:10.228045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:10.230407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:10.230407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:10.230407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:10.230407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:10.230407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:10.230407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:10.230407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:10.230407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:10.230407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:10.230407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:10.230407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:10.230407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:10.230407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:10.230407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:10.230407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:10.230407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:10.230407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:10.232588 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:10.232588 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:10.232588 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:10.232588 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:10.232588 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 284/554] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:10.239945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3016f-c567-427a-bf51-9a593b0579f4 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:10.325033 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52ad09-ec77-4346-92ea-91956f9d4016 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:10.391828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03614e8a-63eb-49b0-845f-a9aa5e93b805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:10.392695 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03614e8a-63eb-49b0-845f-a9aa5e93b805 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:10.393026 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03614e8a-63eb-49b0-845f-a9aa5e93b805 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:10.393805 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03614e8a-63eb-49b0-845f-a9aa5e93b805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:10.405890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03614e8a-63eb-49b0-845f-a9aa5e93b805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:10.408059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03614e8a-63eb-49b0-845f-a9aa5e93b805 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:10.412817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3016f-c567-427a-bf51-9a593b0579f4 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:10.429663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-03614e8a-63eb-49b0-845f-a9aa5e93b805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:10.429663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:10.429663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:10.429663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:10.429663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:10.429663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:10.429663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:10.429663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:10.429663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:10.429663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:10.429663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:10.429663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:10.429663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:10.429663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:10.429663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:10.430686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:10.430686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:10.430686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:10.430686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:10.430686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:10.430686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:10.430686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:10.430686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:10.430686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:10.430686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:10.430686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:10.430686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:10.430686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:10.430686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:10.430686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:10.430686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:10.430686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:10.431521 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:10.431521 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:10.431521 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:10.431521 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:10.431731 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 271/555] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:10.436207 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548a83d-6473-4a0b-8d65-4476b09243c6 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:10.436207 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548a83d-6473-4a0b-8d65-4476b09243c6 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:10.436207 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548a83d-6473-4a0b-8d65-4476b09243c6 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:10.438170 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 272/556] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:10.439030 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548a83d-6473-4a0b-8d65-4476b09243c6 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:10.458806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162885d-c820-4f03-af27-ffdc31746503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:10.459750 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162885d-c820-4f03-af27-ffdc31746503 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:10.460034 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162885d-c820-4f03-af27-ffdc31746503 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:10.460310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162885d-c820-4f03-af27-ffdc31746503 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:10.461218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162885d-c820-4f03-af27-ffdc31746503 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:10.461811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162885d-c820-4f03-af27-ffdc31746503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:10.463221 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162885d-c820-4f03-af27-ffdc31746503 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:10.482161 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 285/557] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:10.484317 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e4c50-1ecd-4211-94e1-7245885d5226 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:10.484607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e4c50-1ecd-4211-94e1-7245885d5226 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:10.484878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e4c50-1ecd-4211-94e1-7245885d5226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:10.506517 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-7d4e4c50-1ecd-4211-94e1-7245885d5226 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:10.508364 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7d4e4c50-1ecd-4211-94e1-7245885d5226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:10.564362 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548a83d-6473-4a0b-8d65-4476b09243c6 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:10.598244 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 286/558] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:10.598244 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e4c50-1ecd-4211-94e1-7245885d5226 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:10.625079 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291af58-8e73-4972-b4bf-b1beb589a13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:10.625743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291af58-8e73-4972-b4bf-b1beb589a13f None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:10.626024 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291af58-8e73-4972-b4bf-b1beb589a13f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:10.626320 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291af58-8e73-4972-b4bf-b1beb589a13f None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:10.627048 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291af58-8e73-4972-b4bf-b1beb589a13f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:10.627352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291af58-8e73-4972-b4bf-b1beb589a13f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:10.628745 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291af58-8e73-4972-b4bf-b1beb589a13f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:10.641585 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 273/559] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:10.646766 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291af58-8e73-4972-b4bf-b1beb589a13f None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:10.743523 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e4c50-1ecd-4211-94e1-7245885d5226 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:10.750234 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291af58-8e73-4972-b4bf-b1beb589a13f None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:10.780663 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241283d-6f00-4daa-820a-c45520cf0c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:10.781448 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241283d-6f00-4daa-820a-c45520cf0c8d None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:10.781728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241283d-6f00-4daa-820a-c45520cf0c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:10.781999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241283d-6f00-4daa-820a-c45520cf0c8d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:10.783214 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d253d-ff12-40cc-8977-8c3d03736b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:10.784684 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d253d-ff12-40cc-8977-8c3d03736b1d None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:10.785077 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d253d-ff12-40cc-8977-8c3d03736b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:10.785456 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d253d-ff12-40cc-8977-8c3d03736b1d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:10.790428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241283d-6f00-4daa-820a-c45520cf0c8d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:10.792150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241283d-6f00-4daa-820a-c45520cf0c8d None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:10.795685 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d253d-ff12-40cc-8977-8c3d03736b1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:10.804025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-834d253d-ff12-40cc-8977-8c3d03736b1d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:10.804025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:10.804025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:10.804025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:10.804025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:10.804025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:10.804025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:10.804025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:10.804025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:10.804025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:10.804025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:10.804025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:10.804025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:10.804025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:10.804025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:10.804025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:10.804898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:10.804898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:10.804898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:10.804898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:10.804898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:10.804898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:10.804898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:10.804898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:10.804898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:10.804898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:10.804898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:10.804898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:10.804898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:10.804898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:10.804898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:10.804898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:10.805694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:10.805694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:10.805694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:10.805694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:10.805694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:10.805694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:10.805694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:10.805694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:10.805694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:10.805694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:10.805694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:10.805694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:10.805694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 274/560] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:10.808261 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d253d-ff12-40cc-8977-8c3d03736b1d None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:10.809956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-3241283d-6f00-4daa-820a-c45520cf0c8d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 10 19:58:10.809956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:10.809956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:10.809956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:10.809956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:10.809956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:10.809956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:10.809956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:10.809956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:10.809956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:10.809956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:10.809956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:10.809956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:10.809956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:10.809956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:10.809956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:10.811342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:10.811342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:10.811342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:10.811342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:10.811342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:10.811342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:10.811342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:10.811342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:10.811342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:10.811342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:10.811342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:10.811342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:10.811342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:10.811342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:10.811342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:10.811342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:10.811342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:10.811342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:10.812465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:10.812465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:10.812465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:10.812465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:10.812465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:10.812465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:10.812465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:10.812465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:10.812465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 10 19:58:10.812465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:10.812465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 287/561] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:58:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:10.812465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441fd02-2130-4eff-9cc6-335db4d6cf69 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:10.812465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441fd02-2130-4eff-9cc6-335db4d6cf69 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:10.813316 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441fd02-2130-4eff-9cc6-335db4d6cf69 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:10.813499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 288/562] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:10.814339 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441fd02-2130-4eff-9cc6-335db4d6cf69 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:10.919076 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441fd02-2130-4eff-9cc6-335db4d6cf69 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:10.922568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d253d-ff12-40cc-8977-8c3d03736b1d None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:10.960765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecf97d-d6ae-4819-871a-a864e2f5ee68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:10.961562 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecf97d-d6ae-4819-871a-a864e2f5ee68 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:10.961848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecf97d-d6ae-4819-871a-a864e2f5ee68 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:10.962151 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecf97d-d6ae-4819-871a-a864e2f5ee68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:10.969157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ecf97d-d6ae-4819-871a-a864e2f5ee68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:10.970705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ecf97d-d6ae-4819-871a-a864e2f5ee68 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:10.979247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8d38a-6c79-4b9b-b7f5-b3160f86accd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:10.979810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8d38a-6c79-4b9b-b7f5-b3160f86accd None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:10.980098 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8d38a-6c79-4b9b-b7f5-b3160f86accd None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:10.980378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8d38a-6c79-4b9b-b7f5-b3160f86accd None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:10.981176 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8d38a-6c79-4b9b-b7f5-b3160f86accd None admin] RBAC: Adding query filter params (name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:10.981573 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8d38a-6c79-4b9b-b7f5-b3160f86accd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:10.983033 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8d38a-6c79-4b9b-b7f5-b3160f86accd None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:10.990452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-d2ecf97d-d6ae-4819-871a-a864e2f5ee68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:10.990452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:10.990452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:10.990452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:10.990452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:10.990452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:10.990452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:10.990452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:10.990452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:10.990452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:10.990452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:10.990452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:10.990452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:10.990452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:10.990452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:10.991522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:10.991522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:10.991522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:10.991522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:10.991522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:10.991522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:10.991522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:10.991522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:10.991522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:10.991522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:10.991522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:10.991522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:10.991522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:10.991522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:10.991522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:10.991522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:10.991522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:10.992429 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:10.992429 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:10.992429 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:10.992429 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:10.992429 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:10.992429 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:10.992429 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:10.992429 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:11.000530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 275/564] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:11.000720 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 289/564] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 19:58:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:11.000720 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8d38a-6c79-4b9b-b7f5-b3160f86accd None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:11.000888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382ee38-8838-4a8a-9929-758eb00e032d None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:11.000888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382ee38-8838-4a8a-9929-758eb00e032d None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:11.000888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382ee38-8838-4a8a-9929-758eb00e032d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:11.016006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-1382ee38-8838-4a8a-9929-758eb00e032d None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:11.017313 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1382ee38-8838-4a8a-9929-758eb00e032d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:11.094108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 276/565] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:11.094108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382ee38-8838-4a8a-9929-758eb00e032d None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:11.107823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8d38a-6c79-4b9b-b7f5-b3160f86accd None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:11.145870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255fd85-b6d7-43be-abbe-59ce0ad99d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:11.146623 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255fd85-b6d7-43be-abbe-59ce0ad99d45 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:11.146950 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255fd85-b6d7-43be-abbe-59ce0ad99d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:11.147200 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255fd85-b6d7-43be-abbe-59ce0ad99d45 None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:11.148122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4255fd85-b6d7-43be-abbe-59ce0ad99d45 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:11.148442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4255fd85-b6d7-43be-abbe-59ce0ad99d45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:11.149933 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4255fd85-b6d7-43be-abbe-59ce0ad99d45 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:11.170598 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 290/566] 10.222.0.65 () {56 vars in 1135 bytes} [Tue May 10 19:58:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:11.170598 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255fd85-b6d7-43be-abbe-59ce0ad99d45 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:11.196983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382ee38-8838-4a8a-9929-758eb00e032d None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:11.229461 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896a246-7ce1-4088-9337-fd9c357cfd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:11.230147 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896a246-7ce1-4088-9337-fd9c357cfd50 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:11.230432 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896a246-7ce1-4088-9337-fd9c357cfd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:11.230707 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896a246-7ce1-4088-9337-fd9c357cfd50 None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:11.231497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896a246-7ce1-4088-9337-fd9c357cfd50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:11.231807 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896a246-7ce1-4088-9337-fd9c357cfd50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:11.233232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896a246-7ce1-4088-9337-fd9c357cfd50 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:11.244622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 277/567] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:11.245408 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896a246-7ce1-4088-9337-fd9c357cfd50 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:11.262650 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255fd85-b6d7-43be-abbe-59ce0ad99d45 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:11.295754 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106c1ba-9a32-4103-a8ed-e80c46f459b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:11.296541 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106c1ba-9a32-4103-a8ed-e80c46f459b3 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:11.296830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106c1ba-9a32-4103-a8ed-e80c46f459b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:11.297108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106c1ba-9a32-4103-a8ed-e80c46f459b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:11.307557 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106c1ba-9a32-4103-a8ed-e80c46f459b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:11.308978 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106c1ba-9a32-4103-a8ed-e80c46f459b3 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:11.326882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-d106c1ba-9a32-4103-a8ed-e80c46f459b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:11.326882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:11.326882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:11.326882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:11.326882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:11.326882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:11.326882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:11.326882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:11.326882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:11.326882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:11.326882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:11.326882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:11.326882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:11.326882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:11.326882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:11.328818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:11.328818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:11.328818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:11.328818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:11.328818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:11.328818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:11.328818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:11.328818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:11.328818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:11.328818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:11.328818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:11.328818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:11.328818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:11.328818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:11.328818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:11.328818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:11.328818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:11.331499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:11.331499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:11.331499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:11.331499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:11.331499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 291/568] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:11.331499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e11381-7521-4cf9-b831-60a9113dd935 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:11.331499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e11381-7521-4cf9-b831-60a9113dd935 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:11.331499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e11381-7521-4cf9-b831-60a9113dd935 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:11.332939 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 292/569] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:11.332939 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e11381-7521-4cf9-b831-60a9113dd935 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:11.388672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896a246-7ce1-4088-9337-fd9c357cfd50 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:11.420582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d44c2-e359-4b6e-bf56-41b20731525a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:11.421424 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d44c2-e359-4b6e-bf56-41b20731525a None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:11.421636 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d44c2-e359-4b6e-bf56-41b20731525a None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:11.421901 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d44c2-e359-4b6e-bf56-41b20731525a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:11.429165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e11381-7521-4cf9-b831-60a9113dd935 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:11.432306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d44c2-e359-4b6e-bf56-41b20731525a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:11.440006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-d72d44c2-e359-4b6e-bf56-41b20731525a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:11.440006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:11.440006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:11.440006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:11.440006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:11.440006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:11.440006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:11.440006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:11.440006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:11.440006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:11.440006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:11.440006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:11.440006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:11.440006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:11.440006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:11.440006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:11.441417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:11.441417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:11.441417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:11.441417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:11.441417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:11.441417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:11.441417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:11.441417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:11.441417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:11.441417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:11.441417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:11.441417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:11.441417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:11.441417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:11.441417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:11.441417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:11.442291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:11.442291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:11.442291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:11.442291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:11.442291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:11.442291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:11.442291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:11.442291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:11.442291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:11.442291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:11.442291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:11.442291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:11.442291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 278/570] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:11.443465 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d44c2-e359-4b6e-bf56-41b20731525a None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:11.466459 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4239a9-5733-47c0-970d-32f837e8e38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:11.467109 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4239a9-5733-47c0-970d-32f837e8e38c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:11.467393 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4239a9-5733-47c0-970d-32f837e8e38c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:11.467667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4239a9-5733-47c0-970d-32f837e8e38c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:11.530442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4239a9-5733-47c0-970d-32f837e8e38c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:11.531905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4239a9-5733-47c0-970d-32f837e8e38c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:11.536628 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d44c2-e359-4b6e-bf56-41b20731525a None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:11.568060 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046bc82b-82ba-4aa6-9cdb-60d3f88fd347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:11.568928 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bc82b-82ba-4aa6-9cdb-60d3f88fd347 None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:11.569207 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bc82b-82ba-4aa6-9cdb-60d3f88fd347 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:11.569482 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bc82b-82ba-4aa6-9cdb-60d3f88fd347 None admin] PATH_INFO: `/v3/projects/a006477d1df94390b599621446bd0f02/users/ae922b2da4814d10bed1fa345700ac04/roles/55528a93cfa64162b2f9574fb5cdde79` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:11.583482 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046bc82b-82ba-4aa6-9cdb-60d3f88fd347 None admin] RBAC: Authorizing `identity:create_grant(project_id=a006477d1df94390b599621446bd0f02, user_id=ae922b2da4814d10bed1fa345700ac04, role_id=55528a93cfa64162b2f9574fb5cdde79)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:11.584913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046bc82b-82ba-4aa6-9cdb-60d3f88fd347 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:11.624045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 293/571] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:58:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:58:11.624848 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4239a9-5733-47c0-970d-32f837e8e38c None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:11.651250 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 279/572] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:58:11 2022] PUT /identity/v3/projects/a006477d1df94390b599621446bd0f02/users/ae922b2da4814d10bed1fa345700ac04/roles/55528a93cfa64162b2f9574fb5cdde79 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:58:11.652520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee25b4f-c9bc-4cda-92c7-b07071324cf4 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:11.652848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee25b4f-c9bc-4cda-92c7-b07071324cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:11.653121 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee25b4f-c9bc-4cda-92c7-b07071324cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:11.670464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-7ee25b4f-c9bc-4cda-92c7-b07071324cf4 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:11.671785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-7ee25b4f-c9bc-4cda-92c7-b07071324cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:11.726486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4239a9-5733-47c0-970d-32f837e8e38c None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:11.758288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 280/573] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:11.759289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee25b4f-c9bc-4cda-92c7-b07071324cf4 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:11.763677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03007137-6bdb-4af7-a393-d48c615928ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:11.764347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03007137-6bdb-4af7-a393-d48c615928ab None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:11.764627 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03007137-6bdb-4af7-a393-d48c615928ab None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:11.764901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03007137-6bdb-4af7-a393-d48c615928ab None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:11.765631 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03007137-6bdb-4af7-a393-d48c615928ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:11.765938 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03007137-6bdb-4af7-a393-d48c615928ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:11.767338 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03007137-6bdb-4af7-a393-d48c615928ab None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:11.779688 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 294/574] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:11.780567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03007137-6bdb-4af7-a393-d48c615928ab None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:11.871059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee25b4f-c9bc-4cda-92c7-b07071324cf4 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:11.886182 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03007137-6bdb-4af7-a393-d48c615928ab None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:11.904573 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774c5b6-0b65-4039-b3f7-e50fe8393f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:11.905901 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774c5b6-0b65-4039-b3f7-e50fe8393f22 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:11.906420 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774c5b6-0b65-4039-b3f7-e50fe8393f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:11.906889 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774c5b6-0b65-4039-b3f7-e50fe8393f22 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:11.907846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774c5b6-0b65-4039-b3f7-e50fe8393f22 None admin] RBAC: Adding query filter params (name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:11.908342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774c5b6-0b65-4039-b3f7-e50fe8393f22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:11.909743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774c5b6-0b65-4039-b3f7-e50fe8393f22 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:11.918039 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 281/575] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 19:58:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:11.918836 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774c5b6-0b65-4039-b3f7-e50fe8393f22 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:11.925389 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff22b5-ea99-4c56-8741-b6dcb633b293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:11.926025 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff22b5-ea99-4c56-8741-b6dcb633b293 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:11.926339 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff22b5-ea99-4c56-8741-b6dcb633b293 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:11.926613 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff22b5-ea99-4c56-8741-b6dcb633b293 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:11.937467 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff22b5-ea99-4c56-8741-b6dcb633b293 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:11.938945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff22b5-ea99-4c56-8741-b6dcb633b293 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:11.956408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-89ff22b5-ea99-4c56-8741-b6dcb633b293 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:11.956408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:11.956408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:11.956408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:11.956408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:11.956408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:11.956408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:11.956408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:11.956408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:11.956408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:11.956408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:11.956408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:11.956408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:11.956408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:11.956408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:11.957312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:11.957312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:11.957312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:11.957312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:11.957312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:11.957312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:11.957312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:11.957312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:11.957312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:11.957312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:11.957312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:11.957312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:11.957312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:11.957312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:11.957312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:11.957312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:11.957312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:11.958247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:11.958247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:11.958247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:11.958247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:11.962210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 295/576] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:11.962811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff22b5-ea99-4c56-8741-b6dcb633b293 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:12.019721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774c5b6-0b65-4039-b3f7-e50fe8393f22 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:12.052894 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac9b4d-33d3-44d4-9203-824fe63047e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:12.053683 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac9b4d-33d3-44d4-9203-824fe63047e1 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:12.053962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac9b4d-33d3-44d4-9203-824fe63047e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:12.054267 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac9b4d-33d3-44d4-9203-824fe63047e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:12.060445 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff22b5-ea99-4c56-8741-b6dcb633b293 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:12.064086 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac9b4d-33d3-44d4-9203-824fe63047e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:12.072246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-c0ac9b4d-33d3-44d4-9203-824fe63047e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:12.072246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:12.072246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:12.072246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:12.072246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:12.072246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:12.072246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:12.072246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:12.072246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:12.072246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:12.072246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:12.072246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:12.072246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:12.072246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:12.072246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:12.072246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:12.073175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:12.073175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:12.073175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:12.073175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:12.073175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:12.073175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:12.073175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:12.073175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:12.073175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:12.073175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:12.073175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:12.073175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:12.073175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:12.073175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:12.073175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:12.073175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:12.079049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:12.079049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:12.079049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:12.079049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:12.079049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:12.079049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:12.079049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:12.079049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:12.079049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:12.079049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:12.079049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:12.079049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:12.079049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 282/577] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:12.079049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac9b4d-33d3-44d4-9203-824fe63047e1 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:12.093798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac806bd-b0f3-45a6-bb74-da266c994890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:12.094614 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac806bd-b0f3-45a6-bb74-da266c994890 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:12.094900 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac806bd-b0f3-45a6-bb74-da266c994890 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:12.095173 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac806bd-b0f3-45a6-bb74-da266c994890 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:12.103619 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac806bd-b0f3-45a6-bb74-da266c994890 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:12.105067 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac806bd-b0f3-45a6-bb74-da266c994890 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:12.119920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-8ac806bd-b0f3-45a6-bb74-da266c994890 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 10 19:58:12.119920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:12.119920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:12.119920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:12.119920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:12.119920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:12.119920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:12.119920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:12.119920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:12.119920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:12.119920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:12.119920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:12.119920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:12.119920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:12.119920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:12.119920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:12.121427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:12.121427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:12.121427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:12.121427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:12.121427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:12.121427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:12.121427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:12.121427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:12.121427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:12.121427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:12.121427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:12.121427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:12.121427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:12.121427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:12.121427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:12.121427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:12.121427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:12.121427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:12.122853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:12.122853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:12.122853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:12.122853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:12.122853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:12.122853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:12.122853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:12.122853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:12.122853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 10 19:58:12.122853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:12.122853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 296/578] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:58:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:12.122853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac806bd-b0f3-45a6-bb74-da266c994890 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:12.182195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac9b4d-33d3-44d4-9203-824fe63047e1 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:12.214974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074193ca-6043-476d-951b-6168f8b4a282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:12.215770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074193ca-6043-476d-951b-6168f8b4a282 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:12.216062 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074193ca-6043-476d-951b-6168f8b4a282 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:12.216349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074193ca-6043-476d-951b-6168f8b4a282 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:12.219147 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac806bd-b0f3-45a6-bb74-da266c994890 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:12.227163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074193ca-6043-476d-951b-6168f8b4a282 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:12.228639 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074193ca-6043-476d-951b-6168f8b4a282 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:12.246352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-074193ca-6043-476d-951b-6168f8b4a282 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:12.246352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:12.246352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:12.246352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:12.246352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:12.246352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:12.246352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:12.246352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:12.246352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:12.246352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:12.246352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:12.246352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:12.246352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:12.246352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:12.246352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:12.255858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:12.255858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:12.255858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:12.255858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:12.255858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:12.255858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:12.255858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:12.255858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:12.255858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:12.255858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:12.255858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:12.255858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:12.255858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:12.255858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:12.255858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:12.255858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:12.255858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:12.256934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:12.256934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:12.256934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:12.256934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:12.256934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 283/579] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:12.256934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074193ca-6043-476d-951b-6168f8b4a282 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:12.258050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7e63f-cbc9-49f3-8af5-5aa4dd00a6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:12.258834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7e63f-cbc9-49f3-8af5-5aa4dd00a6bf None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:12.259548 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7e63f-cbc9-49f3-8af5-5aa4dd00a6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:12.259924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7e63f-cbc9-49f3-8af5-5aa4dd00a6bf None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:12.260759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7e63f-cbc9-49f3-8af5-5aa4dd00a6bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:12.261179 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7e63f-cbc9-49f3-8af5-5aa4dd00a6bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:12.262661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7e63f-cbc9-49f3-8af5-5aa4dd00a6bf None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:12.274712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 297/580] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:12.275499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7e63f-cbc9-49f3-8af5-5aa4dd00a6bf None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:12.347548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074193ca-6043-476d-951b-6168f8b4a282 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:12.373534 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7e63f-cbc9-49f3-8af5-5aa4dd00a6bf None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:12.381355 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad38d4e-85e8-412a-905e-99f6989762f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:12.382671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad38d4e-85e8-412a-905e-99f6989762f6 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:12.383197 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad38d4e-85e8-412a-905e-99f6989762f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:12.383959 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad38d4e-85e8-412a-905e-99f6989762f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:12.385243 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad38d4e-85e8-412a-905e-99f6989762f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:12.385760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad38d4e-85e8-412a-905e-99f6989762f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:12.387927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad38d4e-85e8-412a-905e-99f6989762f6 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:12.399232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 284/581] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 19:58:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:12.400966 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad38d4e-85e8-412a-905e-99f6989762f6 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:12.406667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d1c76-daac-42c6-805f-3fa08abd776f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:12.407378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d1c76-daac-42c6-805f-3fa08abd776f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:12.407657 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d1c76-daac-42c6-805f-3fa08abd776f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:12.407931 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d1c76-daac-42c6-805f-3fa08abd776f None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:12.408798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d1c76-daac-42c6-805f-3fa08abd776f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:12.409618 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d1c76-daac-42c6-805f-3fa08abd776f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:12.411050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d1c76-daac-42c6-805f-3fa08abd776f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:12.436465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 298/582] 10.222.0.65 () {56 vars in 1137 bytes} [Tue May 10 19:58:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:12.436465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d1c76-daac-42c6-805f-3fa08abd776f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:12.503395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad38d4e-85e8-412a-905e-99f6989762f6 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:12.532565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d1c76-daac-42c6-805f-3fa08abd776f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:12.536919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10c09b-4dca-4efb-8452-b997acee10dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:12.537573 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10c09b-4dca-4efb-8452-b997acee10dd None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:12.537854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10c09b-4dca-4efb-8452-b997acee10dd None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:12.538158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10c09b-4dca-4efb-8452-b997acee10dd None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:12.538879 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10c09b-4dca-4efb-8452-b997acee10dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:12.539180 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10c09b-4dca-4efb-8452-b997acee10dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:12.540552 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10c09b-4dca-4efb-8452-b997acee10dd None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:12.556121 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 285/583] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:12.556121 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10c09b-4dca-4efb-8452-b997acee10dd None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:12.570223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31672843-de93-44ec-9536-e627fbad3e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:12.571009 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31672843-de93-44ec-9536-e627fbad3e92 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:12.571294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31672843-de93-44ec-9536-e627fbad3e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:12.571571 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31672843-de93-44ec-9536-e627fbad3e92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:12.578072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31672843-de93-44ec-9536-e627fbad3e92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:12.579535 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31672843-de93-44ec-9536-e627fbad3e92 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:12.593369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-31672843-de93-44ec-9536-e627fbad3e92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:12.593369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:12.593369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:12.593369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:12.593369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:12.593369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:12.593369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:12.593369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:12.593369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:12.593369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:12.593369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:12.593369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:12.593369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:12.593369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:12.593369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:12.594479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:12.594479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:12.594479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:12.594479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:12.594479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:12.594479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:12.594479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:12.594479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:12.594479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:12.594479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:12.594479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:12.594479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:12.594479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:12.594479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:12.594479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:12.594479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:12.594479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:12.595637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:12.595637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:12.595637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:12.595637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:12.595637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:12.595637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:12.595637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:12.595637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:12.598374 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 299/584] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:12.599221 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31672843-de93-44ec-9536-e627fbad3e92 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:12.654034 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10c09b-4dca-4efb-8452-b997acee10dd None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:12.687758 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a893527-e567-40a2-8fed-91e10236d3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:12.688710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a893527-e567-40a2-8fed-91e10236d3fe None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:12.689067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a893527-e567-40a2-8fed-91e10236d3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:12.689352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a893527-e567-40a2-8fed-91e10236d3fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:12.698713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31672843-de93-44ec-9536-e627fbad3e92 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:12.699567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a893527-e567-40a2-8fed-91e10236d3fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:12.701006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a893527-e567-40a2-8fed-91e10236d3fe None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:12.718913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-2a893527-e567-40a2-8fed-91e10236d3fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:12.718913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:12.718913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:12.718913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:12.718913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:12.718913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:12.718913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:12.718913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:12.718913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:12.718913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:12.718913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:12.718913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:12.718913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:12.718913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:12.718913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:12.719896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:12.719896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:12.719896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:12.719896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:12.719896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:12.719896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:12.719896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:12.719896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:12.719896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:12.719896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:12.719896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:12.719896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:12.719896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:12.719896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:12.719896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:12.719896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:12.719896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:12.720784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:12.720784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:12.720784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:12.720784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:12.720784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 286/585] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:12.721062 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a893527-e567-40a2-8fed-91e10236d3fe None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:12.738256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a928c-12f2-442a-b04d-59fec845bf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:12.739045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a928c-12f2-442a-b04d-59fec845bf5f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:12.739332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a928c-12f2-442a-b04d-59fec845bf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:12.739608 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a928c-12f2-442a-b04d-59fec845bf5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:12.750216 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a928c-12f2-442a-b04d-59fec845bf5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:12.751666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a928c-12f2-442a-b04d-59fec845bf5f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:12.770858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-7d7a928c-12f2-442a-b04d-59fec845bf5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:12.770858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:12.770858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:12.770858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:12.770858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:12.770858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:12.770858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:12.770858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:12.770858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:12.770858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:12.770858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:12.770858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:12.770858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:12.770858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:12.770858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:12.771876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:12.771876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:12.771876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:12.771876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:12.771876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:12.771876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:12.771876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:12.771876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:12.771876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:12.771876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:12.771876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:12.771876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:12.771876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:12.771876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:12.771876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:12.771876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:12.771876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:12.772769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:12.772769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:12.772769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:12.772769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:12.772769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 300/586] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:12.773049 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a928c-12f2-442a-b04d-59fec845bf5f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:12.825988 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a893527-e567-40a2-8fed-91e10236d3fe None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:12.859668 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55206708-73ae-47f3-b01a-06a041b0ae82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:12.860650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55206708-73ae-47f3-b01a-06a041b0ae82 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:12.860932 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55206708-73ae-47f3-b01a-06a041b0ae82 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:12.861206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55206708-73ae-47f3-b01a-06a041b0ae82 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:12.868616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55206708-73ae-47f3-b01a-06a041b0ae82 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:12.870062 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55206708-73ae-47f3-b01a-06a041b0ae82 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:12.872463 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a928c-12f2-442a-b04d-59fec845bf5f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:12.888227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-55206708-73ae-47f3-b01a-06a041b0ae82 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 10 19:58:12.888227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:12.888227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:12.888227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:12.888227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:12.888227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:12.888227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:12.888227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:12.888227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:12.888227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:12.888227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:12.888227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:12.888227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:12.888227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:12.888227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:12.888227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:12.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:12.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:12.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:12.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:12.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:12.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:12.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:12.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:12.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:12.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:12.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:12.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:12.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:12.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:12.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:12.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:12.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:12.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:12.890685 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:12.890685 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:12.890685 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:12.890685 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:12.890685 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:12.890685 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:12.890685 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:12.890685 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:12.890685 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 10 19:58:12.890685 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:12.895600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 287/587] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:12.895600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55206708-73ae-47f3-b01a-06a041b0ae82 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:12.912895 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba36c5-cdc2-45b8-9ddb-5dfb1d9f93a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:12.913555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba36c5-cdc2-45b8-9ddb-5dfb1d9f93a8 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:12.913838 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba36c5-cdc2-45b8-9ddb-5dfb1d9f93a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:12.914153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba36c5-cdc2-45b8-9ddb-5dfb1d9f93a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:12.914951 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba36c5-cdc2-45b8-9ddb-5dfb1d9f93a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:12.915264 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba36c5-cdc2-45b8-9ddb-5dfb1d9f93a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:12.916689 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba36c5-cdc2-45b8-9ddb-5dfb1d9f93a8 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:12.933409 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 301/588] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:12.933409 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba36c5-cdc2-45b8-9ddb-5dfb1d9f93a8 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:13.002374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55206708-73ae-47f3-b01a-06a041b0ae82 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:13.035660 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeecfd9e-a0d4-4c4b-b073-71778fa9f603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:13.036318 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeecfd9e-a0d4-4c4b-b073-71778fa9f603 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:13.036602 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeecfd9e-a0d4-4c4b-b073-71778fa9f603 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:13.036881 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeecfd9e-a0d4-4c4b-b073-71778fa9f603 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:13.037607 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeecfd9e-a0d4-4c4b-b073-71778fa9f603 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:13.037915 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeecfd9e-a0d4-4c4b-b073-71778fa9f603 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:13.038227 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba36c5-cdc2-45b8-9ddb-5dfb1d9f93a8 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:13.039312 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeecfd9e-a0d4-4c4b-b073-71778fa9f603 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:13.050620 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 288/589] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:13.051475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeecfd9e-a0d4-4c4b-b073-71778fa9f603 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:13.076630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce61b7-8026-4ab0-9fc1-91547b7fc00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:13.077537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce61b7-8026-4ab0-9fc1-91547b7fc00d None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:13.078070 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce61b7-8026-4ab0-9fc1-91547b7fc00d None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:13.078446 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce61b7-8026-4ab0-9fc1-91547b7fc00d None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:13.079180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce61b7-8026-4ab0-9fc1-91547b7fc00d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:13.079488 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce61b7-8026-4ab0-9fc1-91547b7fc00d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:13.081370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce61b7-8026-4ab0-9fc1-91547b7fc00d None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:13.096872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 302/590] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:13.096872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce61b7-8026-4ab0-9fc1-91547b7fc00d None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:13.159336 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeecfd9e-a0d4-4c4b-b073-71778fa9f603 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:13.192381 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3305912-f667-4ed1-b7d5-32623e66f028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:13.193052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3305912-f667-4ed1-b7d5-32623e66f028 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:13.193342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3305912-f667-4ed1-b7d5-32623e66f028 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:13.193620 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3305912-f667-4ed1-b7d5-32623e66f028 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:13.194497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3305912-f667-4ed1-b7d5-32623e66f028 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:13.194812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3305912-f667-4ed1-b7d5-32623e66f028 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:13.196249 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3305912-f667-4ed1-b7d5-32623e66f028 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:13.198326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce61b7-8026-4ab0-9fc1-91547b7fc00d None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:13.218445 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 289/591] 10.222.0.65 () {56 vars in 1139 bytes} [Tue May 10 19:58:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:13.218445 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3305912-f667-4ed1-b7d5-32623e66f028 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:13.238623 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5e9aa-44dd-4c51-81b8-d25994bdfdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:13.239362 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5e9aa-44dd-4c51-81b8-d25994bdfdeb None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:13.239723 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5e9aa-44dd-4c51-81b8-d25994bdfdeb None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:13.240258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5e9aa-44dd-4c51-81b8-d25994bdfdeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:13.241100 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5e9aa-44dd-4c51-81b8-d25994bdfdeb None admin] RBAC: Adding query filter params (role.id=55528a93cfa64162b2f9574fb5cdde79, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=b4d10b258a53484da22950e2b1db2e41) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:13.241412 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5e9aa-44dd-4c51-81b8-d25994bdfdeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:13.242862 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5e9aa-44dd-4c51-81b8-d25994bdfdeb None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:13.259777 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 303/592] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:58:13 2022] GET /identity/v3/role_assignments?user.id=b4d10b258a53484da22950e2b1db2e41&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=55528a93cfa64162b2f9574fb5cdde79 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:13.259777 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5e9aa-44dd-4c51-81b8-d25994bdfdeb None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:13.333644 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3305912-f667-4ed1-b7d5-32623e66f028 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:13.371803 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10911fe4-21d1-4089-98de-107da1867e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:13.372568 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5e9aa-44dd-4c51-81b8-d25994bdfdeb None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:13.372884 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10911fe4-21d1-4089-98de-107da1867e96 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:13.373269 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10911fe4-21d1-4089-98de-107da1867e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:13.373643 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10911fe4-21d1-4089-98de-107da1867e96 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:13.382462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10911fe4-21d1-4089-98de-107da1867e96 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:13.384091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10911fe4-21d1-4089-98de-107da1867e96 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:13.400156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-10911fe4-21d1-4089-98de-107da1867e96 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 10 19:58:13.400156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:13.400156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:13.400156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:13.400156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:13.400156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:13.400156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:13.400156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:13.400156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:13.400156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:13.400156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:13.400156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:13.400156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:13.400156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:13.400156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:13.400156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:13.401780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:13.401780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:13.401780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:13.401780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:13.401780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:13.401780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:13.401780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:13.401780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:13.401780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:13.401780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:13.401780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:13.401780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:13.401780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:13.401780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:13.401780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:13.401780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:13.401780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:13.401780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:13.403054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:13.403054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:13.403054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:13.403054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:13.403054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:13.403054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:13.403054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:13.403054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:13.403054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 10 19:58:13.403054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:13.408506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 290/593] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:58:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:13.408506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10911fe4-21d1-4089-98de-107da1867e96 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:13.416237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45efd72-d877-4b8e-b0bc-b953b71e9c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:13.416237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45efd72-d877-4b8e-b0bc-b953b71e9c5e None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:13.416237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45efd72-d877-4b8e-b0bc-b953b71e9c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:13.416237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45efd72-d877-4b8e-b0bc-b953b71e9c5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:13.426037 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45efd72-d877-4b8e-b0bc-b953b71e9c5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:13.427940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45efd72-d877-4b8e-b0bc-b953b71e9c5e None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:13.444674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-e45efd72-d877-4b8e-b0bc-b953b71e9c5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:13.444674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:13.444674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:13.444674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:13.444674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:13.444674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:13.444674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:13.444674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:13.444674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:13.444674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:13.444674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:13.444674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:13.444674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:13.444674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:13.444674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:13.445640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:13.445640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:13.445640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:13.445640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:13.445640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:13.445640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:13.445640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:13.445640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:13.445640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:13.445640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:13.445640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:13.445640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:13.445640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:13.445640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:13.445640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:13.445640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:13.445640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:13.446544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:13.446544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:13.446544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:13.446544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:13.446544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:13.446544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:13.446544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:13.446544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:13.453893 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 304/594] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:13.453893 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45efd72-d877-4b8e-b0bc-b953b71e9c5e None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:13.540456 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10911fe4-21d1-4089-98de-107da1867e96 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:13.577590 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45efd72-d877-4b8e-b0bc-b953b71e9c5e None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:13.580595 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899763a4-de41-4279-93c4-d294513fcfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:13.581662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899763a4-de41-4279-93c4-d294513fcfdd None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:13.582239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899763a4-de41-4279-93c4-d294513fcfdd None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:13.582712 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899763a4-de41-4279-93c4-d294513fcfdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:13.593350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899763a4-de41-4279-93c4-d294513fcfdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:13.595148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899763a4-de41-4279-93c4-d294513fcfdd None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:13.615078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d380ed-b262-4361-8c48-f3bfaacb6d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:13.615616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-899763a4-de41-4279-93c4-d294513fcfdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:13.615616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:13.615616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:13.615616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:13.615616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:13.615616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:13.615616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:13.615616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:13.615616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:13.615616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:13.615616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:13.615616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:13.615616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:13.615616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:13.615616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:13.616454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:13.616454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:13.616454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:13.616454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:13.616454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:13.616454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:13.616454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:13.616454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:13.616454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:13.616454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:13.616454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:13.616454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:13.616454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:13.616454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:13.616454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:13.616454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:13.616454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:13.617364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:13.617364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:13.617364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:13.617364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:13.617587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d380ed-b262-4361-8c48-f3bfaacb6d99 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:13.617587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d380ed-b262-4361-8c48-f3bfaacb6d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:13.617587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d380ed-b262-4361-8c48-f3bfaacb6d99 None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:13.617587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d380ed-b262-4361-8c48-f3bfaacb6d99 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:13.617587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d380ed-b262-4361-8c48-f3bfaacb6d99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:13.618869 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d380ed-b262-4361-8c48-f3bfaacb6d99 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:13.624380 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 291/595] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:13.625464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899763a4-de41-4279-93c4-d294513fcfdd None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:13.641565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 305/596] 10.222.0.65 () {56 vars in 1137 bytes} [Tue May 10 19:58:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:13.641565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d380ed-b262-4361-8c48-f3bfaacb6d99 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:13.735644 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899763a4-de41-4279-93c4-d294513fcfdd None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:13.755368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d380ed-b262-4361-8c48-f3bfaacb6d99 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:13.771650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a3316-ac83-45ae-b967-677bbded0a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:13.772680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a3316-ac83-45ae-b967-677bbded0a64 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:13.773198 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a3316-ac83-45ae-b967-677bbded0a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:13.773712 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a3316-ac83-45ae-b967-677bbded0a64 None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:13.774736 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a3316-ac83-45ae-b967-677bbded0a64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:13.775250 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a3316-ac83-45ae-b967-677bbded0a64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:13.776867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a3316-ac83-45ae-b967-677bbded0a64 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:13.795039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c5331-3b2e-48d7-b188-69018179f31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:13.795039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c5331-3b2e-48d7-b188-69018179f31f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:13.795039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c5331-3b2e-48d7-b188-69018179f31f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:13.795039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c5331-3b2e-48d7-b188-69018179f31f None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:13.795426 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c5331-3b2e-48d7-b188-69018179f31f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:13.795426 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c5331-3b2e-48d7-b188-69018179f31f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:13.795426 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c5331-3b2e-48d7-b188-69018179f31f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:13.796379 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 292/597] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:13.797461 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a3316-ac83-45ae-b967-677bbded0a64 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:13.810770 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 306/598] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:13.810770 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c5331-3b2e-48d7-b188-69018179f31f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:13.903052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a3316-ac83-45ae-b967-677bbded0a64 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:13.910613 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c5331-3b2e-48d7-b188-69018179f31f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:13.937730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433849b7-27e9-4de0-a7ea-3fbae68b5b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:13.938551 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433849b7-27e9-4de0-a7ea-3fbae68b5b7d None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:13.938831 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433849b7-27e9-4de0-a7ea-3fbae68b5b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:13.939107 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433849b7-27e9-4de0-a7ea-3fbae68b5b7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:13.943900 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a316969-f9ad-4d53-8e60-20755ae23a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:13.944786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a316969-f9ad-4d53-8e60-20755ae23a36 None admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:13.945072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a316969-f9ad-4d53-8e60-20755ae23a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:13.945346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a316969-f9ad-4d53-8e60-20755ae23a36 None admin] PATH_INFO: `/v3/projects/a006477d1df94390b599621446bd0f02/users/0e366ec240e5464d8c5e92f807123c23/roles/55528a93cfa64162b2f9574fb5cdde79` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:13.950011 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433849b7-27e9-4de0-a7ea-3fbae68b5b7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:13.951572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433849b7-27e9-4de0-a7ea-3fbae68b5b7d None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:13.960002 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a316969-f9ad-4d53-8e60-20755ae23a36 None admin] RBAC: Authorizing `identity:create_grant(project_id=a006477d1df94390b599621446bd0f02, user_id=0e366ec240e5464d8c5e92f807123c23, role_id=55528a93cfa64162b2f9574fb5cdde79)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:13.961501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a316969-f9ad-4d53-8e60-20755ae23a36 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:13.968835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-433849b7-27e9-4de0-a7ea-3fbae68b5b7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:13.968835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:13.968835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:13.968835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:13.968835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:13.968835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:13.968835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:13.968835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:13.968835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:13.968835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:13.968835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:13.968835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:13.968835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:13.968835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:13.968835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:13.969763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:13.969763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:13.969763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:13.969763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:13.969763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:13.969763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:13.969763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:13.969763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:13.969763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:13.969763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:13.969763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:13.969763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:13.969763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:13.969763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:13.969763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:13.969763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:13.969763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:13.970774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:13.970774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:13.970774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:13.970774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:13.970774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 293/599] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:13.971067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433849b7-27e9-4de0-a7ea-3fbae68b5b7d None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:14.066140 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433849b7-27e9-4de0-a7ea-3fbae68b5b7d None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:14.091848 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 307/600] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:58:13 2022] PUT /identity/v3/projects/a006477d1df94390b599621446bd0f02/users/0e366ec240e5464d8c5e92f807123c23/roles/55528a93cfa64162b2f9574fb5cdde79 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:58:14.093509 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4bb89-36e2-4bb7-8658-a057d49ae476 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:14.094014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4bb89-36e2-4bb7-8658-a057d49ae476 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:14.094452 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4bb89-36e2-4bb7-8658-a057d49ae476 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:14.096367 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 308/601] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:14.098057 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4bb89-36e2-4bb7-8658-a057d49ae476 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:14.108850 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326f218-9b89-4959-bed4-8cc416094bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:14.109581 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326f218-9b89-4959-bed4-8cc416094bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:14.109872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326f218-9b89-4959-bed4-8cc416094bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:14.110198 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326f218-9b89-4959-bed4-8cc416094bc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:14.118764 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326f218-9b89-4959-bed4-8cc416094bc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:14.120292 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326f218-9b89-4959-bed4-8cc416094bc5 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:14.136248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-6326f218-9b89-4959-bed4-8cc416094bc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:14.136248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:14.136248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:14.136248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:14.136248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:14.136248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:14.136248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:14.136248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:14.136248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:14.136248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:14.136248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:14.136248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:14.136248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:14.136248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:14.136248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:14.137421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:14.137421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:14.137421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:14.137421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:14.137421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:14.137421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:14.137421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:14.137421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:14.137421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:14.137421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:14.137421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:14.137421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:14.137421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:14.137421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:14.137421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:14.137421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:14.137421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:14.143781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:14.143781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:14.143781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:14.143781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:14.143781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:14.143781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:14.143781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:14.143781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:14.143781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 294/602] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:14.143781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416d9d2-076d-4752-9f0a-9184855ea928 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:14.143781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416d9d2-076d-4752-9f0a-9184855ea928 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:14.144629 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416d9d2-076d-4752-9f0a-9184855ea928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:14.160777 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-b416d9d2-076d-4752-9f0a-9184855ea928 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:14.162218 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b416d9d2-076d-4752-9f0a-9184855ea928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:14.221177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4bb89-36e2-4bb7-8658-a057d49ae476 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:14.238671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 295/603] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:14.240007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416d9d2-076d-4752-9f0a-9184855ea928 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:14.259753 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c926b0a-3623-4b8f-9a5d-49e636f615e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:14.260753 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c926b0a-3623-4b8f-9a5d-49e636f615e0 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:14.261177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c926b0a-3623-4b8f-9a5d-49e636f615e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:14.261592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c926b0a-3623-4b8f-9a5d-49e636f615e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:14.262588 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c926b0a-3623-4b8f-9a5d-49e636f615e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:14.263052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c926b0a-3623-4b8f-9a5d-49e636f615e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:14.264801 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c926b0a-3623-4b8f-9a5d-49e636f615e0 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:14.279245 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 309/604] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:14.280349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c926b0a-3623-4b8f-9a5d-49e636f615e0 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:14.374231 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416d9d2-076d-4752-9f0a-9184855ea928 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:14.407696 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c926b0a-3623-4b8f-9a5d-49e636f615e0 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:14.420364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370b6ff-f19f-4fb5-a0c4-0f4f9e4abd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:14.421556 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370b6ff-f19f-4fb5-a0c4-0f4f9e4abd92 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:14.421997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370b6ff-f19f-4fb5-a0c4-0f4f9e4abd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:14.422946 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370b6ff-f19f-4fb5-a0c4-0f4f9e4abd92 None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:14.423926 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370b6ff-f19f-4fb5-a0c4-0f4f9e4abd92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:14.424459 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370b6ff-f19f-4fb5-a0c4-0f4f9e4abd92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:14.426183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370b6ff-f19f-4fb5-a0c4-0f4f9e4abd92 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:14.446554 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 296/605] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:14.452484 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370b6ff-f19f-4fb5-a0c4-0f4f9e4abd92 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:14.462833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9661e-9428-48f6-93c1-ef47cc4c4cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:14.463624 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9661e-9428-48f6-93c1-ef47cc4c4cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:14.463911 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9661e-9428-48f6-93c1-ef47cc4c4cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:14.464233 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9661e-9428-48f6-93c1-ef47cc4c4cb5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:14.477531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9661e-9428-48f6-93c1-ef47cc4c4cb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:14.488110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-0fe9661e-9428-48f6-93c1-ef47cc4c4cb5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:14.488110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:14.488110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:14.488110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:14.488110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:14.488110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:14.488110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:14.488110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:14.488110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:14.488110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:14.488110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:14.488110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:14.488110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:14.488110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:14.488110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:14.488110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:14.489262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:14.489262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:14.489262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:14.489262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:14.489262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:14.489262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:14.489262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:14.489262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:14.489262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:14.489262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:14.489262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:14.489262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:14.489262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:14.489262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:14.489262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:14.489262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:14.490549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:14.490549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:14.490549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:14.490549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:14.490549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:14.490549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:14.490549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:14.490549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:14.490549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:14.490549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:14.490549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:14.490549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:14.490549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 310/606] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:14.494812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9661e-9428-48f6-93c1-ef47cc4c4cb5 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:14.556427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370b6ff-f19f-4fb5-a0c4-0f4f9e4abd92 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:14.599699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1def9c5-da40-4955-9975-3bc3ab82c473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:14.600493 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1def9c5-da40-4955-9975-3bc3ab82c473 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:14.600779 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1def9c5-da40-4955-9975-3bc3ab82c473 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:14.601486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1def9c5-da40-4955-9975-3bc3ab82c473 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:14.610726 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1def9c5-da40-4955-9975-3bc3ab82c473 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:14.612548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1def9c5-da40-4955-9975-3bc3ab82c473 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:14.613059 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9661e-9428-48f6-93c1-ef47cc4c4cb5 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:14.631549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-d1def9c5-da40-4955-9975-3bc3ab82c473 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:14.631549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:14.631549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:14.631549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:14.631549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:14.631549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:14.631549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:14.631549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:14.631549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:14.631549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:14.631549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:14.631549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:14.631549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:14.631549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:14.631549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:14.632640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:14.632640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:14.632640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:14.632640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:14.632640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:14.632640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:14.632640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:14.632640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:14.632640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:14.632640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:14.632640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:14.632640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:14.632640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:14.632640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:14.632640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:14.632640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:14.632640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:14.633739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:14.633739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:14.633739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:14.633739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:14.633739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:14.633739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:14.633739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:14.633739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:14.638167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 297/607] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:14.639050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1def9c5-da40-4955-9975-3bc3ab82c473 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:14.658254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdf9e7-f474-42e8-994f-328f49ad1271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:14.659234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdf9e7-f474-42e8-994f-328f49ad1271 None admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:14.659586 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdf9e7-f474-42e8-994f-328f49ad1271 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:14.659914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdf9e7-f474-42e8-994f-328f49ad1271 None admin] PATH_INFO: `/v3/projects/a006477d1df94390b599621446bd0f02/users/a6aabd1571d04bcfae842626fef5ed6d/roles/55528a93cfa64162b2f9574fb5cdde79` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:14.675711 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbdf9e7-f474-42e8-994f-328f49ad1271 None admin] RBAC: Authorizing `identity:create_grant(project_id=a006477d1df94390b599621446bd0f02, user_id=a6aabd1571d04bcfae842626fef5ed6d, role_id=55528a93cfa64162b2f9574fb5cdde79)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:14.677271 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbdf9e7-f474-42e8-994f-328f49ad1271 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:14.759515 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1def9c5-da40-4955-9975-3bc3ab82c473 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:14.780168 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 311/608] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:58:14 2022] PUT /identity/v3/projects/a006477d1df94390b599621446bd0f02/users/a6aabd1571d04bcfae842626fef5ed6d/roles/55528a93cfa64162b2f9574fb5cdde79 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:58:14.781274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdf9e7-f474-42e8-994f-328f49ad1271 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:14.819246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e5773-73e5-43e0-a4ad-848edc74b583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:14.819987 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e5773-73e5-43e0-a4ad-848edc74b583 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:14.820271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e5773-73e5-43e0-a4ad-848edc74b583 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:14.820604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e5773-73e5-43e0-a4ad-848edc74b583 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:14.821466 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e5773-73e5-43e0-a4ad-848edc74b583 None admin] RBAC: Adding query filter params (name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:14.821726 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e5773-73e5-43e0-a4ad-848edc74b583 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:14.823192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e5773-73e5-43e0-a4ad-848edc74b583 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:14.842648 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 298/609] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 19:58:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:14.848206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e5773-73e5-43e0-a4ad-848edc74b583 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:14.895738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdf9e7-f474-42e8-994f-328f49ad1271 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:14.931957 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9aba99-77ce-4373-813f-767531011bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:14.933028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9aba99-77ce-4373-813f-767531011bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:14.933512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9aba99-77ce-4373-813f-767531011bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:14.934002 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9aba99-77ce-4373-813f-767531011bd0 None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:14.935118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9aba99-77ce-4373-813f-767531011bd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:14.935664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9aba99-77ce-4373-813f-767531011bd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:14.937349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9aba99-77ce-4373-813f-767531011bd0 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:14.950927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 312/610] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:14.957760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9aba99-77ce-4373-813f-767531011bd0 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:14.992396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e5773-73e5-43e0-a4ad-848edc74b583 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:15.085519 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0662ab-7d2a-44a4-9133-c6e9b2c3d487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:15.086541 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0662ab-7d2a-44a4-9133-c6e9b2c3d487 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:15.086925 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0662ab-7d2a-44a4-9133-c6e9b2c3d487 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:15.087298 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0662ab-7d2a-44a4-9133-c6e9b2c3d487 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:15.104134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0662ab-7d2a-44a4-9133-c6e9b2c3d487 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:15.105693 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0662ab-7d2a-44a4-9133-c6e9b2c3d487 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:15.109136 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9aba99-77ce-4373-813f-767531011bd0 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:15.125316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-fd0662ab-7d2a-44a4-9133-c6e9b2c3d487 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:15.125316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:15.125316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:15.125316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:15.125316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:15.125316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:15.125316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:15.125316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:15.125316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:15.125316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:15.125316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:15.125316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:15.125316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:15.125316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:15.125316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:15.126501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:15.126501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:15.126501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:15.126501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:15.126501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:15.126501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:15.126501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:15.126501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:15.126501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:15.126501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:15.126501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:15.126501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:15.126501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:15.126501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:15.126501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:15.126501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:15.126501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:15.128101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:15.128101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:15.128101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:15.128101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:15.130253 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 299/611] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:15.135053 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0662ab-7d2a-44a4-9133-c6e9b2c3d487 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:15.151029 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942e5b7-e685-47b2-b274-4ae5d779a6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:15.152666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942e5b7-e685-47b2-b274-4ae5d779a6e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:15.153121 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942e5b7-e685-47b2-b274-4ae5d779a6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:15.153819 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942e5b7-e685-47b2-b274-4ae5d779a6e2 None admin] PATH_INFO: `/v3/projects/a006477d1df94390b599621446bd0f02/users/67a252cde71a455fb1dc4bf9c47c0490/roles/55528a93cfa64162b2f9574fb5cdde79` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:15.175231 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1942e5b7-e685-47b2-b274-4ae5d779a6e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a006477d1df94390b599621446bd0f02, user_id=67a252cde71a455fb1dc4bf9c47c0490, role_id=55528a93cfa64162b2f9574fb5cdde79)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:15.176966 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1942e5b7-e685-47b2-b274-4ae5d779a6e2 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:15.248374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0662ab-7d2a-44a4-9133-c6e9b2c3d487 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:15.259719 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 313/612] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:58:14 2022] PUT /identity/v3/projects/a006477d1df94390b599621446bd0f02/users/67a252cde71a455fb1dc4bf9c47c0490/roles/55528a93cfa64162b2f9574fb5cdde79 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:58:15.302704 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80ac13-e0c4-46f0-8775-ecfadebed07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:15.303703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80ac13-e0c4-46f0-8775-ecfadebed07e None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:15.304158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80ac13-e0c4-46f0-8775-ecfadebed07e None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:15.304620 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80ac13-e0c4-46f0-8775-ecfadebed07e None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:15.305526 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a80ac13-e0c4-46f0-8775-ecfadebed07e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:15.306011 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a80ac13-e0c4-46f0-8775-ecfadebed07e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:15.307649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a80ac13-e0c4-46f0-8775-ecfadebed07e None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:15.326760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 300/613] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:15.333537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942e5b7-e685-47b2-b274-4ae5d779a6e2 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:15.451047 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942e5b7-e685-47b2-b274-4ae5d779a6e2 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:15.493487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbd376-d64f-45ec-8aef-d24ce4106e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:15.494315 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbd376-d64f-45ec-8aef-d24ce4106e4c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:15.494600 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbd376-d64f-45ec-8aef-d24ce4106e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:15.494878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbd376-d64f-45ec-8aef-d24ce4106e4c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:15.502840 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bbd376-d64f-45ec-8aef-d24ce4106e4c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:15.504278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bbd376-d64f-45ec-8aef-d24ce4106e4c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:15.526133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-02bbd376-d64f-45ec-8aef-d24ce4106e4c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 19:58:15.526133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:15.526133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:15.526133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:15.526133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:15.526133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:15.526133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:15.526133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:15.526133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:15.526133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:15.526133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:15.526133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:15.526133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:15.526133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:15.526133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:15.526133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:15.527166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:15.527166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:15.527166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:15.527166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:15.527166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:15.527166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:15.527166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:15.527166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:15.527166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:15.527166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:15.527166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:15.527166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:15.527166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:15.527166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:15.527166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:15.527166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:15.527166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:15.527166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:15.528113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:15.528113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:15.528113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:15.528113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:15.528113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:15.528113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:15.528113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:15.528113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:15.528113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 19:58:15.528113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:15.528113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 314/614] 10.222.0.65 () {56 vars in 1093 bytes} [Tue May 10 19:58:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:15.535689 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80ac13-e0c4-46f0-8775-ecfadebed07e None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:15.646801 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80ac13-e0c4-46f0-8775-ecfadebed07e None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:15.686995 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d24d8a-dcf1-4467-8459-e1484d1c9c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:15.687597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d24d8a-dcf1-4467-8459-e1484d1c9c89 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:15.687879 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d24d8a-dcf1-4467-8459-e1484d1c9c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:15.688155 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d24d8a-dcf1-4467-8459-e1484d1c9c89 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:15.689008 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d24d8a-dcf1-4467-8459-e1484d1c9c89 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:15.689321 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d24d8a-dcf1-4467-8459-e1484d1c9c89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:15.690806 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d24d8a-dcf1-4467-8459-e1484d1c9c89 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:15.709605 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 301/615] 10.222.0.65 () {56 vars in 1133 bytes} [Tue May 10 19:58:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:15.715174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbd376-d64f-45ec-8aef-d24ce4106e4c None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:15.746203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd0785-4d08-4d38-b2a0-0ae40550f797 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:15.746509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd0785-4d08-4d38-b2a0-0ae40550f797 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:15.746773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd0785-4d08-4d38-b2a0-0ae40550f797 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:15.748379 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 302/616] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:15.753264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fff0b1-8716-455a-bf46-3e8a81c8da02 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:15.753552 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fff0b1-8716-455a-bf46-3e8a81c8da02 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:15.753832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fff0b1-8716-455a-bf46-3e8a81c8da02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:15.770705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-94fff0b1-8716-455a-bf46-3e8a81c8da02 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:15.771895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-94fff0b1-8716-455a-bf46-3e8a81c8da02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:15.848587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 303/617] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:58:15.874256 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fff0b1-8716-455a-bf46-3e8a81c8da02 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:15.944076 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbd376-d64f-45ec-8aef-d24ce4106e4c None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:15.982652 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fff0b1-8716-455a-bf46-3e8a81c8da02 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:16.020612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1186868-74ae-4378-adbb-a08699948500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:16.021420 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1186868-74ae-4378-adbb-a08699948500 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:16.021704 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1186868-74ae-4378-adbb-a08699948500 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:16.021981 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1186868-74ae-4378-adbb-a08699948500 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:16.033361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1186868-74ae-4378-adbb-a08699948500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:16.041333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-f1186868-74ae-4378-adbb-a08699948500 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:16.041333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:16.041333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:16.041333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:16.041333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:16.041333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:16.041333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:16.041333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:16.041333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:16.041333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:16.041333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:16.041333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:16.041333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:16.041333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:16.041333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:16.041333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:16.042264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:16.042264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:16.042264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:16.042264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:16.042264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:16.042264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:16.042264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:16.042264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:16.042264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:16.042264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:16.042264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:16.042264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:16.042264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:16.042264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:16.042264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:16.042264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:16.043196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:16.043196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:16.043196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:16.043196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:16.043196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:16.043196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:16.043196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:16.043196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:16.043196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:16.043196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:16.043196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:16.043196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:16.043196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 304/618] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:16.045111 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196aaa1-41b0-43ba-9cf6-22479841d115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:16.045990 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196aaa1-41b0-43ba-9cf6-22479841d115 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:16.046498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196aaa1-41b0-43ba-9cf6-22479841d115 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:16.046980 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196aaa1-41b0-43ba-9cf6-22479841d115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:16.054105 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1186868-74ae-4378-adbb-a08699948500 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:16.183381 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196aaa1-41b0-43ba-9cf6-22479841d115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:16.184851 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196aaa1-41b0-43ba-9cf6-22479841d115 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:16.193099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1186868-74ae-4378-adbb-a08699948500 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:16.268085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cea01-6ca2-48bd-9cb8-e7c243b88241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:16.268085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cea01-6ca2-48bd-9cb8-e7c243b88241 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:16.268085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cea01-6ca2-48bd-9cb8-e7c243b88241 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:16.268085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cea01-6ca2-48bd-9cb8-e7c243b88241 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:16.268598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60cea01-6ca2-48bd-9cb8-e7c243b88241 None admin] RBAC: Adding query filter params (name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:16.268598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60cea01-6ca2-48bd-9cb8-e7c243b88241 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:16.269416 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60cea01-6ca2-48bd-9cb8-e7c243b88241 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:16.281888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 305/619] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 19:58:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:16.287630 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cea01-6ca2-48bd-9cb8-e7c243b88241 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:16.338561 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 315/620] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:58:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:58:16.340795 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196aaa1-41b0-43ba-9cf6-22479841d115 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:16.393363 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cea01-6ca2-48bd-9cb8-e7c243b88241 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:16.429269 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8397f-b717-47b6-ad55-cf51c1253f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:16.429269 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8397f-b717-47b6-ad55-cf51c1253f42 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:16.429269 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8397f-b717-47b6-ad55-cf51c1253f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:16.429269 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8397f-b717-47b6-ad55-cf51c1253f42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:16.440470 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8397f-b717-47b6-ad55-cf51c1253f42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:16.441900 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8397f-b717-47b6-ad55-cf51c1253f42 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:16.459492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-e6e8397f-b717-47b6-ad55-cf51c1253f42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:16.459492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:16.459492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:16.459492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:16.459492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:16.459492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:16.459492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:16.459492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:16.459492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:16.459492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:16.459492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:16.459492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:16.459492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:16.459492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:16.459492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:16.460407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:16.460407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:16.460407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:16.460407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:16.460407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:16.460407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:16.460407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:16.460407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:16.460407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:16.460407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:16.460407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:16.460407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:16.460407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:16.460407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:16.460407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:16.460407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:16.460407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:16.461284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:16.461284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:16.461284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:16.461284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:16.461284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 306/621] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:16.466507 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8397f-b717-47b6-ad55-cf51c1253f42 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:16.520310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196aaa1-41b0-43ba-9cf6-22479841d115 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:16.567717 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8397f-b717-47b6-ad55-cf51c1253f42 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:16.582480 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2598a-efb9-41cc-977e-447e7ef3b62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:16.583944 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2598a-efb9-41cc-977e-447e7ef3b62a None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:16.584398 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2598a-efb9-41cc-977e-447e7ef3b62a None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:16.585208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2598a-efb9-41cc-977e-447e7ef3b62a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:16.598570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2598a-efb9-41cc-977e-447e7ef3b62a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:16.600608 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2598a-efb9-41cc-977e-447e7ef3b62a None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:16.606214 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee08fa9-bb1c-45ec-b5d3-9fa482773843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:16.606964 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee08fa9-bb1c-45ec-b5d3-9fa482773843 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:16.607330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee08fa9-bb1c-45ec-b5d3-9fa482773843 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:16.607682 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee08fa9-bb1c-45ec-b5d3-9fa482773843 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:16.608536 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee08fa9-bb1c-45ec-b5d3-9fa482773843 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:16.609341 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee08fa9-bb1c-45ec-b5d3-9fa482773843 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:16.611183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee08fa9-bb1c-45ec-b5d3-9fa482773843 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:16.622676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-0bf2598a-efb9-41cc-977e-447e7ef3b62a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:16.622676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:16.622676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:16.622676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:16.622676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:16.622676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:16.622676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:16.622676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:16.622676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:16.622676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:16.622676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:16.622676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:16.622676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:16.622676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:16.622676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:16.623762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:16.623762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:16.623762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:16.623762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:16.623762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:16.623762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:16.623762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:16.623762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:16.623762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:16.623762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:16.623762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:16.623762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:16.623762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:16.623762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:16.623762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:16.623762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:16.623762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:16.624938 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:16.624938 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:16.624938 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:16.624938 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:16.630672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 307/622] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:16.631153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 316/623] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:16.632163 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2598a-efb9-41cc-977e-447e7ef3b62a None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:16.632530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5e606-61e6-4bbf-bb2f-6805de7b83f0 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:16.632892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5e606-61e6-4bbf-bb2f-6805de7b83f0 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:16.633243 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5e606-61e6-4bbf-bb2f-6805de7b83f0 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:16.634976 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 308/624] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:16.638168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5e606-61e6-4bbf-bb2f-6805de7b83f0 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:16.755556 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5e606-61e6-4bbf-bb2f-6805de7b83f0 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:16.756630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2598a-efb9-41cc-977e-447e7ef3b62a None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:16.790965 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1dcaf-0ae7-4b4d-b429-c88d14332261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:16.791781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1dcaf-0ae7-4b4d-b429-c88d14332261 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:16.792139 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1dcaf-0ae7-4b4d-b429-c88d14332261 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:16.792488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1dcaf-0ae7-4b4d-b429-c88d14332261 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:16.797870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89346da-499c-400b-b0a5-b4824b4a3716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:16.798609 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89346da-499c-400b-b0a5-b4824b4a3716 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:16.799048 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89346da-499c-400b-b0a5-b4824b4a3716 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:16.799519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89346da-499c-400b-b0a5-b4824b4a3716 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:16.799992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1dcaf-0ae7-4b4d-b429-c88d14332261 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:16.800251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89346da-499c-400b-b0a5-b4824b4a3716 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:16.800556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89346da-499c-400b-b0a5-b4824b4a3716 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:16.801661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1dcaf-0ae7-4b4d-b429-c88d14332261 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:16.801994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89346da-499c-400b-b0a5-b4824b4a3716 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:16.816934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-45b1dcaf-0ae7-4b4d-b429-c88d14332261 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 10 19:58:16.816934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:16.816934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:16.816934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:16.816934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:16.816934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:16.816934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:16.816934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:16.816934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:16.816934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:16.816934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:16.816934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:16.816934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:16.816934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:16.816934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:16.816934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:16.818167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:16.818167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:16.818167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:16.818167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:16.818167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:16.818167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:16.818167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:16.818167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:16.818167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:16.818167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:16.818167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:16.818167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:16.818167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:16.818167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:16.818167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:16.818167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:16.818167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:16.818167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:16.819338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:16.819338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:16.819338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:16.819338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:16.819338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:16.819338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:16.819338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:16.819338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:16.819338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 10 19:58:16.819338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:16.821485 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 317/625] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:16.821485 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3a612-527d-4c6a-b714-e948f38f4871 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:16.824996 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3a612-527d-4c6a-b714-e948f38f4871 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:16.824996 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3a612-527d-4c6a-b714-e948f38f4871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:16.825352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 309/626] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:58:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:16.830225 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1dcaf-0ae7-4b4d-b429-c88d14332261 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:16.854109 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-2df3a612-527d-4c6a-b714-e948f38f4871 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:16.854109 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-2df3a612-527d-4c6a-b714-e948f38f4871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:16.939745 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 318/627] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:16.946114 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3a612-527d-4c6a-b714-e948f38f4871 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:17.005474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1dcaf-0ae7-4b4d-b429-c88d14332261 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:17.040681 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f19d03-ed80-481d-8309-8a00575762f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:17.041527 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f19d03-ed80-481d-8309-8a00575762f8 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:17.041765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f19d03-ed80-481d-8309-8a00575762f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:17.042045 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f19d03-ed80-481d-8309-8a00575762f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:17.049050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f19d03-ed80-481d-8309-8a00575762f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:17.051076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f19d03-ed80-481d-8309-8a00575762f8 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:17.070104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-30f19d03-ed80-481d-8309-8a00575762f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:17.070104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:17.070104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:17.070104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:17.070104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:17.070104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:17.070104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:17.070104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:17.070104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:17.070104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:17.070104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:17.070104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:17.070104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:17.070104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:17.070104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:17.071032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:17.071032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:17.071032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:17.071032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:17.071032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:17.071032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:17.071032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:17.071032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:17.071032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:17.071032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:17.071032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:17.071032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:17.071032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:17.071032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:17.071032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:17.071032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:17.071032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:17.071929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:17.071929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:17.071929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:17.071929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:17.071929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:17.071929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:17.071929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:17.071929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:17.071929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 310/628] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:17.071929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f19d03-ed80-481d-8309-8a00575762f8 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:17.072461 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3a612-527d-4c6a-b714-e948f38f4871 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:17.118127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a3d4e-304b-451c-bc85-dc0cba42b44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:17.118127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a3d4e-304b-451c-bc85-dc0cba42b44f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:17.118127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a3d4e-304b-451c-bc85-dc0cba42b44f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:17.118127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a3d4e-304b-451c-bc85-dc0cba42b44f None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:17.118694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a3d4e-304b-451c-bc85-dc0cba42b44f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:17.118694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a3d4e-304b-451c-bc85-dc0cba42b44f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:17.118694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a3d4e-304b-451c-bc85-dc0cba42b44f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:17.138122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 319/629] 10.222.0.65 () {56 vars in 1135 bytes} [Tue May 10 19:58:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:17.138122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a3d4e-304b-451c-bc85-dc0cba42b44f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:17.239369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a3d4e-304b-451c-bc85-dc0cba42b44f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:17.261732 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f19d03-ed80-481d-8309-8a00575762f8 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:17.276455 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e584e77-15a8-406f-8026-90e6fd269d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:17.277197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e584e77-15a8-406f-8026-90e6fd269d51 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:17.277478 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e584e77-15a8-406f-8026-90e6fd269d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:17.277930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e584e77-15a8-406f-8026-90e6fd269d51 None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:17.282108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e584e77-15a8-406f-8026-90e6fd269d51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:17.282108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e584e77-15a8-406f-8026-90e6fd269d51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:17.282108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e584e77-15a8-406f-8026-90e6fd269d51 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:17.314156 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 320/630] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:17.314156 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e584e77-15a8-406f-8026-90e6fd269d51 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:17.320716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6958f1a-be88-4bb9-9924-fe35ac1fbcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:17.320716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6958f1a-be88-4bb9-9924-fe35ac1fbcfa None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:17.320716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6958f1a-be88-4bb9-9924-fe35ac1fbcfa None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:17.320716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6958f1a-be88-4bb9-9924-fe35ac1fbcfa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:17.332793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6958f1a-be88-4bb9-9924-fe35ac1fbcfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:17.342530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-d6958f1a-be88-4bb9-9924-fe35ac1fbcfa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:17.342530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:17.342530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:17.342530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:17.342530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:17.342530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:17.342530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:17.342530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:17.342530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:17.342530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:17.342530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:17.342530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:17.342530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:17.342530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:17.342530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:17.342530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:17.343500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:17.343500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:17.343500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:17.343500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:17.343500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:17.343500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:17.343500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:17.343500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:17.343500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:17.343500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:17.343500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:17.343500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:17.343500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:17.343500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:17.343500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:17.343500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:17.344393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:17.344393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:17.344393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:17.344393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:17.344393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:17.344393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:17.344393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:17.344393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:17.344393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:17.344393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:17.344393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:17.344393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:17.345062 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 311/631] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:17.345616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6958f1a-be88-4bb9-9924-fe35ac1fbcfa None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:17.469651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6958f1a-be88-4bb9-9924-fe35ac1fbcfa None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:17.477153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e584e77-15a8-406f-8026-90e6fd269d51 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:17.515730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0dc2b-7c84-4375-aaa2-09ba595454df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:17.516380 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0dc2b-7c84-4375-aaa2-09ba595454df None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:17.516668 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0dc2b-7c84-4375-aaa2-09ba595454df None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:17.516946 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0dc2b-7c84-4375-aaa2-09ba595454df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:17.517791 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0dc2b-7c84-4375-aaa2-09ba595454df None admin] RBAC: Adding query filter params (role.id=55528a93cfa64162b2f9574fb5cdde79, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=ae922b2da4814d10bed1fa345700ac04) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:17.518786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0dc2b-7c84-4375-aaa2-09ba595454df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:17.520236 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0dc2b-7c84-4375-aaa2-09ba595454df None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:17.523332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7df075-92bb-40a7-ba44-f013441d5870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:17.524320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7df075-92bb-40a7-ba44-f013441d5870 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:17.524818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7df075-92bb-40a7-ba44-f013441d5870 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:17.525281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7df075-92bb-40a7-ba44-f013441d5870 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:17.526276 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 312/632] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:58:17 2022] GET /identity/v3/role_assignments?user.id=ae922b2da4814d10bed1fa345700ac04&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=55528a93cfa64162b2f9574fb5cdde79 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:17.526895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0dc2b-7c84-4375-aaa2-09ba595454df None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:17.546491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7df075-92bb-40a7-ba44-f013441d5870 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:17.547915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7df075-92bb-40a7-ba44-f013441d5870 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:17.566828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-3e7df075-92bb-40a7-ba44-f013441d5870 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:17.566828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:17.566828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:17.566828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:17.566828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:17.566828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:17.566828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:17.566828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:17.566828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:17.566828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:17.566828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:17.566828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:17.566828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:17.566828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:17.566828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:17.567875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:17.567875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:17.567875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:17.567875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:17.567875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:17.567875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:17.567875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:17.567875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:17.567875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:17.567875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:17.567875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:17.567875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:17.567875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:17.567875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:17.567875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:17.567875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:17.567875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:17.568780 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:17.568780 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:17.568780 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:17.568780 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:17.568780 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 321/633] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:17.574132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4e1f2-0428-4351-b849-a5a36d5bdf5b None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:17.574433 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4e1f2-0428-4351-b849-a5a36d5bdf5b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:17.574712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4e1f2-0428-4351-b849-a5a36d5bdf5b None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:17.580133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 322/634] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:17.580133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4e1f2-0428-4351-b849-a5a36d5bdf5b None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:17.658148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0dc2b-7c84-4375-aaa2-09ba595454df None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:17.691254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4e1f2-0428-4351-b849-a5a36d5bdf5b None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:17.694767 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cd17f-5297-411c-a3c1-1e76b553f418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:17.695448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cd17f-5297-411c-a3c1-1e76b553f418 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:17.695734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cd17f-5297-411c-a3c1-1e76b553f418 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:17.696013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cd17f-5297-411c-a3c1-1e76b553f418 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:17.696784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cd17f-5297-411c-a3c1-1e76b553f418 None admin] RBAC: Adding query filter params (name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:17.697152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cd17f-5297-411c-a3c1-1e76b553f418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:17.698596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cd17f-5297-411c-a3c1-1e76b553f418 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:17.712888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 313/635] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 19:58:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:17.714394 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd2f78-e086-4ddf-8d6a-769c3da41ff9 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:17.714693 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd2f78-e086-4ddf-8d6a-769c3da41ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:17.714972 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd2f78-e086-4ddf-8d6a-769c3da41ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:17.733575 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32883a9c-43e6-41df-b4e5-51713415c394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:17.734508 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32883a9c-43e6-41df-b4e5-51713415c394 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:17.735038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32883a9c-43e6-41df-b4e5-51713415c394 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:17.735324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-47fd2f78-e086-4ddf-8d6a-769c3da41ff9 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:17.735795 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32883a9c-43e6-41df-b4e5-51713415c394 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:17.736525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-47fd2f78-e086-4ddf-8d6a-769c3da41ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:17.737440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32883a9c-43e6-41df-b4e5-51713415c394 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:17.737939 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32883a9c-43e6-41df-b4e5-51713415c394 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:17.740873 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32883a9c-43e6-41df-b4e5-51713415c394 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:17.757629 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 323/636] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:17.760573 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32883a9c-43e6-41df-b4e5-51713415c394 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:17.817488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 314/637] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:17.818655 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd2f78-e086-4ddf-8d6a-769c3da41ff9 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:17.878868 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32883a9c-43e6-41df-b4e5-51713415c394 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:17.925142 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee23d14-4ae9-4376-b87f-e30dfbec0967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:17.926278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee23d14-4ae9-4376-b87f-e30dfbec0967 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:17.926565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee23d14-4ae9-4376-b87f-e30dfbec0967 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:17.926841 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee23d14-4ae9-4376-b87f-e30dfbec0967 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:17.940704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee23d14-4ae9-4376-b87f-e30dfbec0967 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:17.941453 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd2f78-e086-4ddf-8d6a-769c3da41ff9 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:17.942172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee23d14-4ae9-4376-b87f-e30dfbec0967 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:17.964195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-5ee23d14-4ae9-4376-b87f-e30dfbec0967 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:17.964195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:17.964195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:17.964195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:17.964195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:17.964195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:17.964195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:17.964195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:17.964195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:17.964195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:17.964195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:17.964195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:17.964195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:17.964195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:17.964195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:17.965150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:17.965150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:17.965150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:17.965150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:17.965150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:17.965150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:17.965150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:17.965150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:17.965150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:17.965150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:17.965150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:17.965150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:17.965150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:17.965150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:17.965150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:17.965150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:17.965150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:17.966041 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:17.966041 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:17.966041 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:17.966041 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:17.966041 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 324/638] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:17.966811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee23d14-4ae9-4376-b87f-e30dfbec0967 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:17.991273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b09786-ef6f-49f5-bc4c-3effca8960fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:17.992089 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b09786-ef6f-49f5-bc4c-3effca8960fd None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:17.992361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b09786-ef6f-49f5-bc4c-3effca8960fd None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:17.992638 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b09786-ef6f-49f5-bc4c-3effca8960fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:18.002882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b09786-ef6f-49f5-bc4c-3effca8960fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:18.004621 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b09786-ef6f-49f5-bc4c-3effca8960fd None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:18.022474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-78b09786-ef6f-49f5-bc4c-3effca8960fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:18.022474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:18.022474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:18.022474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:18.022474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:18.022474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:18.022474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:18.022474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:18.022474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:18.022474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:18.022474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:18.022474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:18.022474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:18.022474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:18.022474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:18.023439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:18.023439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:18.023439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:18.023439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:18.023439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:18.023439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:18.023439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:18.023439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:18.023439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:18.023439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:18.023439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:18.023439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:18.023439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:18.023439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:18.023439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:18.023439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:18.023439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:18.024318 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:18.024318 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:18.024318 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:18.024318 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:18.024318 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:18.024318 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:18.024318 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:18.024318 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:18.029059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 315/639] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:18.029986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b09786-ef6f-49f5-bc4c-3effca8960fd None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:18.087890 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee23d14-4ae9-4376-b87f-e30dfbec0967 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:18.122542 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e9ab4-2b05-4dd8-88dc-438b6ede44c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:18.123338 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e9ab4-2b05-4dd8-88dc-438b6ede44c0 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:18.123614 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e9ab4-2b05-4dd8-88dc-438b6ede44c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:18.123887 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e9ab4-2b05-4dd8-88dc-438b6ede44c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:18.132242 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b09786-ef6f-49f5-bc4c-3effca8960fd None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:18.133683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e9ab4-2b05-4dd8-88dc-438b6ede44c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:18.141289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-8b5e9ab4-2b05-4dd8-88dc-438b6ede44c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:18.141289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:18.141289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:18.141289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:18.141289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:18.141289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:18.141289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:18.141289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:18.141289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:18.141289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:18.141289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:18.141289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:18.141289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:18.141289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:18.141289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:18.141289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:18.142346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:18.142346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:18.142346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:18.142346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:18.142346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:18.142346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:18.142346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:18.142346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:18.142346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:18.142346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:18.142346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:18.142346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:18.142346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:18.142346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:18.142346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:18.142346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:18.143332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:18.143332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:18.143332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:18.143332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:18.143332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:18.143332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:18.143332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:18.143332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:18.143332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:18.143332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:18.143332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:18.143332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:18.143332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 325/640] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:18.143332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e9ab4-2b05-4dd8-88dc-438b6ede44c0 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:18.170447 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebef601-e976-4e40-884b-ec28fdcb2cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:18.171108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebef601-e976-4e40-884b-ec28fdcb2cdd None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:18.171391 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebef601-e976-4e40-884b-ec28fdcb2cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:18.171667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebef601-e976-4e40-884b-ec28fdcb2cdd None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:18.172393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebef601-e976-4e40-884b-ec28fdcb2cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:18.172697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebef601-e976-4e40-884b-ec28fdcb2cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:18.174060 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebef601-e976-4e40-884b-ec28fdcb2cdd None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:18.191144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 316/641] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:18.191144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3557a1a-6d59-4788-8cd2-760733d71523 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:18.191144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3557a1a-6d59-4788-8cd2-760733d71523 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:18.191144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3557a1a-6d59-4788-8cd2-760733d71523 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:18.191144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 317/642] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:18.191144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3557a1a-6d59-4788-8cd2-760733d71523 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:18.249445 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e9ab4-2b05-4dd8-88dc-438b6ede44c0 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:18.282427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d3dde-f8e1-419c-b7bf-90850648c8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:18.283098 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d3dde-f8e1-419c-b7bf-90850648c8d4 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:18.283383 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d3dde-f8e1-419c-b7bf-90850648c8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:18.284168 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d3dde-f8e1-419c-b7bf-90850648c8d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:18.284967 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d3dde-f8e1-419c-b7bf-90850648c8d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:18.285281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d3dde-f8e1-419c-b7bf-90850648c8d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:18.286808 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d3dde-f8e1-419c-b7bf-90850648c8d4 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:18.293072 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3557a1a-6d59-4788-8cd2-760733d71523 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:18.299823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 326/643] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:18.300652 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d3dde-f8e1-419c-b7bf-90850648c8d4 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:18.332182 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0115fc1-754d-4570-92b7-5d08d4996b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:18.332865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0115fc1-754d-4570-92b7-5d08d4996b18 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:18.333136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0115fc1-754d-4570-92b7-5d08d4996b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:18.333567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0115fc1-754d-4570-92b7-5d08d4996b18 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:18.334190 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0115fc1-754d-4570-92b7-5d08d4996b18 None admin] RBAC: Adding query filter params (name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:18.334500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0115fc1-754d-4570-92b7-5d08d4996b18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:18.335915 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0115fc1-754d-4570-92b7-5d08d4996b18 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:18.344080 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 318/644] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 19:58:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:18.345249 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f002c1-1ea0-4e97-8578-981d69dcad1f None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:18.345539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f002c1-1ea0-4e97-8578-981d69dcad1f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:18.345815 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f002c1-1ea0-4e97-8578-981d69dcad1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:18.363304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-08f002c1-1ea0-4e97-8578-981d69dcad1f None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:18.364476 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-08f002c1-1ea0-4e97-8578-981d69dcad1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:18.406912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d3dde-f8e1-419c-b7bf-90850648c8d4 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:18.442811 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 319/645] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:18.443653 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f002c1-1ea0-4e97-8578-981d69dcad1f None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:18.457331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5225357-3fe9-4317-a025-a3a562421001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:18.458556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5225357-3fe9-4317-a025-a3a562421001 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:18.459388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5225357-3fe9-4317-a025-a3a562421001 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:18.459812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5225357-3fe9-4317-a025-a3a562421001 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:18.468663 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5225357-3fe9-4317-a025-a3a562421001 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:18.470154 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5225357-3fe9-4317-a025-a3a562421001 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:18.484952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-c5225357-3fe9-4317-a025-a3a562421001 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 10 19:58:18.484952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:18.484952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:18.484952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:18.484952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:18.484952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:18.484952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:18.484952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:18.484952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:18.484952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:18.484952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:18.484952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:18.484952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:18.484952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:18.484952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:18.484952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:18.486630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:18.486630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:18.486630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:18.486630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:18.486630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:18.486630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:18.486630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:18.486630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:18.486630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:18.486630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:18.486630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:18.486630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:18.486630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:18.486630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:18.486630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:18.486630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:18.486630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:18.486630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:18.487776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:18.487776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:18.487776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:18.487776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:18.487776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:18.487776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:18.487776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:18.487776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:18.487776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 10 19:58:18.487776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:18.487776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 327/646] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:58:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:18.487776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5225357-3fe9-4317-a025-a3a562421001 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:18.548440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f002c1-1ea0-4e97-8578-981d69dcad1f None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:18.581925 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cd2f1-30b7-491e-bf33-a995f1fefa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:18.582832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cd2f1-30b7-491e-bf33-a995f1fefa99 None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:18.583108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cd2f1-30b7-491e-bf33-a995f1fefa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:18.583380 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cd2f1-30b7-491e-bf33-a995f1fefa99 None admin] PATH_INFO: `/v3/projects/a006477d1df94390b599621446bd0f02/users/b4d10b258a53484da22950e2b1db2e41/roles/55528a93cfa64162b2f9574fb5cdde79` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:18.586872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5225357-3fe9-4317-a025-a3a562421001 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:18.597569 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030cd2f1-30b7-491e-bf33-a995f1fefa99 None admin] RBAC: Authorizing `identity:create_grant(project_id=a006477d1df94390b599621446bd0f02, user_id=b4d10b258a53484da22950e2b1db2e41, role_id=55528a93cfa64162b2f9574fb5cdde79)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:18.598970 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030cd2f1-30b7-491e-bf33-a995f1fefa99 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:18.620325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004aac6-ed3f-4beb-acb2-c9a7a8abfc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:18.621104 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004aac6-ed3f-4beb-acb2-c9a7a8abfc30 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:18.621382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004aac6-ed3f-4beb-acb2-c9a7a8abfc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:18.621652 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004aac6-ed3f-4beb-acb2-c9a7a8abfc30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:18.631494 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004aac6-ed3f-4beb-acb2-c9a7a8abfc30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:18.632912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004aac6-ed3f-4beb-acb2-c9a7a8abfc30 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:18.650116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-7004aac6-ed3f-4beb-acb2-c9a7a8abfc30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:18.650116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:18.650116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:18.650116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:18.650116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:18.650116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:18.650116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:18.650116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:18.650116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:18.650116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:18.650116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:18.650116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:18.650116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:18.650116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:18.650116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:18.651170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:18.651170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:18.651170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:18.651170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:18.651170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:18.651170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:18.651170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:18.651170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:18.651170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:18.651170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:18.651170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:18.651170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:18.651170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:18.651170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:18.651170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:18.651170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:18.651170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:18.652107 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:18.652107 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:18.652107 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:18.652107 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:18.652107 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 328/647] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:18.652439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004aac6-ed3f-4beb-acb2-c9a7a8abfc30 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:18.682112 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 320/648] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:58:18 2022] PUT /identity/v3/projects/a006477d1df94390b599621446bd0f02/users/b4d10b258a53484da22950e2b1db2e41/roles/55528a93cfa64162b2f9574fb5cdde79 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:58:18.682112 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cd2f1-30b7-491e-bf33-a995f1fefa99 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:18.790040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cd2f1-30b7-491e-bf33-a995f1fefa99 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:18.797213 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004aac6-ed3f-4beb-acb2-c9a7a8abfc30 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:18.827841 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664706d-7d59-4d8a-9066-7e2e7018a0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:18.828575 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664706d-7d59-4d8a-9066-7e2e7018a0aa None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:18.828867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664706d-7d59-4d8a-9066-7e2e7018a0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:18.829144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664706d-7d59-4d8a-9066-7e2e7018a0aa None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:18.830139 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664706d-7d59-4d8a-9066-7e2e7018a0aa None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:18.831023 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664706d-7d59-4d8a-9066-7e2e7018a0aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:18.832483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664706d-7d59-4d8a-9066-7e2e7018a0aa None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:18.841326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8c9dd-870b-49da-b6c6-0a8ab45fb2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:18.842246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8c9dd-870b-49da-b6c6-0a8ab45fb2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:18.842598 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8c9dd-870b-49da-b6c6-0a8ab45fb2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:18.842941 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8c9dd-870b-49da-b6c6-0a8ab45fb2d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:18.852916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 321/649] 10.222.0.65 () {56 vars in 1137 bytes} [Tue May 10 19:58:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:18.853836 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664706d-7d59-4d8a-9066-7e2e7018a0aa None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:18.855417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8c9dd-870b-49da-b6c6-0a8ab45fb2d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:18.863814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-89d8c9dd-870b-49da-b6c6-0a8ab45fb2d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:18.863814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:18.863814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:18.863814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:18.863814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:18.863814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:18.863814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:18.863814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:18.863814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:18.863814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:18.863814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:18.863814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:18.863814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:18.863814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:18.863814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:18.863814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:18.865081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:18.865081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:18.865081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:18.865081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:18.865081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:18.865081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:18.865081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:18.865081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:18.865081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:18.865081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:18.865081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:18.865081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:18.865081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:18.865081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:18.865081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:18.865081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:18.865949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:18.865949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:18.865949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:18.865949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:18.865949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:18.865949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:18.865949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:18.865949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:18.865949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:18.865949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:18.865949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:18.865949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:18.870279 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 329/650] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:18.871052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8c9dd-870b-49da-b6c6-0a8ab45fb2d3 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:18.960374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664706d-7d59-4d8a-9066-7e2e7018a0aa None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:18.986766 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8c9dd-870b-49da-b6c6-0a8ab45fb2d3 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:18.995383 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5363222-bb5c-4e5f-b03c-b45ecfc69493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:18.996341 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5363222-bb5c-4e5f-b03c-b45ecfc69493 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:18.996832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5363222-bb5c-4e5f-b03c-b45ecfc69493 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:18.997358 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5363222-bb5c-4e5f-b03c-b45ecfc69493 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:18.998315 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5363222-bb5c-4e5f-b03c-b45ecfc69493 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:18.998825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5363222-bb5c-4e5f-b03c-b45ecfc69493 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:19.000395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5363222-bb5c-4e5f-b03c-b45ecfc69493 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:19.012715 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 322/651] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:19.013803 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5363222-bb5c-4e5f-b03c-b45ecfc69493 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:19.034861 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e2241-aaeb-4714-b62c-73c665566a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:19.035928 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e2241-aaeb-4714-b62c-73c665566a7c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:19.036437 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e2241-aaeb-4714-b62c-73c665566a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:19.036934 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e2241-aaeb-4714-b62c-73c665566a7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:19.105643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e2241-aaeb-4714-b62c-73c665566a7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:19.107465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e2241-aaeb-4714-b62c-73c665566a7c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:19.138873 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5363222-bb5c-4e5f-b03c-b45ecfc69493 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:19.171966 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6b25d-5821-4201-9790-4e95f118bf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:19.172633 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6b25d-5821-4201-9790-4e95f118bf8a None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:19.172912 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6b25d-5821-4201-9790-4e95f118bf8a None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:19.173182 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6b25d-5821-4201-9790-4e95f118bf8a None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:19.173925 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6b25d-5821-4201-9790-4e95f118bf8a None admin] RBAC: Adding query filter params (name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:19.174400 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6b25d-5821-4201-9790-4e95f118bf8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:19.176678 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6b25d-5821-4201-9790-4e95f118bf8a None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:19.190910 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 323/652] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 19:58:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:19.191788 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6b25d-5821-4201-9790-4e95f118bf8a None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:19.211016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 330/653] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:58:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:58:19.216497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e2241-aaeb-4714-b62c-73c665566a7c None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:19.294608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6b25d-5821-4201-9790-4e95f118bf8a None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:19.317788 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e2241-aaeb-4714-b62c-73c665566a7c None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:19.329363 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c37012-8fc2-4ebb-a1b1-14f0da3213d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:19.330286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c37012-8fc2-4ebb-a1b1-14f0da3213d9 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:19.330503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c37012-8fc2-4ebb-a1b1-14f0da3213d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:19.330775 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c37012-8fc2-4ebb-a1b1-14f0da3213d9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:19.338545 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c37012-8fc2-4ebb-a1b1-14f0da3213d9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:19.340221 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c37012-8fc2-4ebb-a1b1-14f0da3213d9 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:19.354164 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977483c8-c872-4f49-bfff-013631520fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:19.354802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-69c37012-8fc2-4ebb-a1b1-14f0da3213d9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 10 19:58:19.354802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:19.354802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:19.354802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:19.354802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:19.354802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:19.354802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:19.354802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:19.354802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:19.354802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:19.354802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:19.354802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:19.354802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:19.354802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:19.354802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:19.354802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:19.355782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:19.355782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:19.355782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:19.355782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:19.355782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:19.355782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:19.355782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:19.355782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:19.355782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:19.355782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:19.355782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:19.355782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:19.355782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:19.355782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:19.355782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:19.355782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:19.355782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:19.355782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:19.356765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:19.356765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:19.356765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:19.356765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:19.356765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:19.356765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:19.356765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:19.356765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:19.356765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 10 19:58:19.356765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:19.358368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977483c8-c872-4f49-bfff-013631520fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:19.358892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977483c8-c872-4f49-bfff-013631520fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:19.359365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977483c8-c872-4f49-bfff-013631520fb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:19.360720 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 324/654] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:19.362125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c37012-8fc2-4ebb-a1b1-14f0da3213d9 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:19.372007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977483c8-c872-4f49-bfff-013631520fb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:19.373817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977483c8-c872-4f49-bfff-013631520fb2 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:19.393143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-977483c8-c872-4f49-bfff-013631520fb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:19.393143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:19.393143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:19.393143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:19.393143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:19.393143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:19.393143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:19.393143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:19.393143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:19.393143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:19.393143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:19.393143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:19.393143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:19.393143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:19.393143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:19.394172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:19.394172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:19.394172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:19.394172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:19.394172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:19.394172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:19.394172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:19.394172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:19.394172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:19.394172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:19.394172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:19.394172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:19.394172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:19.394172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:19.394172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:19.394172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:19.394172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:19.395079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:19.395079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:19.395079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:19.395079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:19.397366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 331/655] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:19.398479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977483c8-c872-4f49-bfff-013631520fb2 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:19.479004 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c37012-8fc2-4ebb-a1b1-14f0da3213d9 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:19.509950 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977483c8-c872-4f49-bfff-013631520fb2 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:19.514728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e67376-bfc8-4538-b39e-99c5ce099981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:19.515514 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e67376-bfc8-4538-b39e-99c5ce099981 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:19.515793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e67376-bfc8-4538-b39e-99c5ce099981 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:19.516070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e67376-bfc8-4538-b39e-99c5ce099981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:19.527577 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e67376-bfc8-4538-b39e-99c5ce099981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:19.529041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e67376-bfc8-4538-b39e-99c5ce099981 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:19.546866 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86b7a5-1aff-4ac3-b5a1-ee976f369675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:19.547063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-44e67376-bfc8-4538-b39e-99c5ce099981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:19.547063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:19.547063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:19.547063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:19.547063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:19.547063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:19.547063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:19.547063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:19.547063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:19.547063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:19.547063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:19.547063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:19.547063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:19.547063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:19.547063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:19.547927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:19.547927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:19.547927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:19.547927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:19.547927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:19.547927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:19.547927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:19.547927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:19.547927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:19.547927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:19.547927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:19.547927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:19.547927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:19.547927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:19.547927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:19.547927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:19.547927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:19.548813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:19.548813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:19.548813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:19.548813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:19.550870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86b7a5-1aff-4ac3-b5a1-ee976f369675 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:19.551398 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86b7a5-1aff-4ac3-b5a1-ee976f369675 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:19.551879 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86b7a5-1aff-4ac3-b5a1-ee976f369675 None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:19.553213 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 325/656] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:19.553459 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86b7a5-1aff-4ac3-b5a1-ee976f369675 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:19.553987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86b7a5-1aff-4ac3-b5a1-ee976f369675 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:19.554114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e67376-bfc8-4538-b39e-99c5ce099981 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:19.555823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86b7a5-1aff-4ac3-b5a1-ee976f369675 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:19.575309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 332/657] 10.222.0.65 () {56 vars in 1139 bytes} [Tue May 10 19:58:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:19.576567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86b7a5-1aff-4ac3-b5a1-ee976f369675 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:19.657477 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e67376-bfc8-4538-b39e-99c5ce099981 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:19.679880 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86b7a5-1aff-4ac3-b5a1-ee976f369675 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:19.691005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45062d1d-8cd0-495c-8711-041c3bbdec63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:19.691662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45062d1d-8cd0-495c-8711-041c3bbdec63 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:19.691941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45062d1d-8cd0-495c-8711-041c3bbdec63 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:19.692213 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45062d1d-8cd0-495c-8711-041c3bbdec63 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:19.692953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45062d1d-8cd0-495c-8711-041c3bbdec63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:19.693242 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45062d1d-8cd0-495c-8711-041c3bbdec63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:19.694673 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45062d1d-8cd0-495c-8711-041c3bbdec63 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:19.711155 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 326/658] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:19.712020 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45062d1d-8cd0-495c-8711-041c3bbdec63 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:19.718777 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e519f8-ce1f-48e9-9634-4cdbf8d74168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:19.719700 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e519f8-ce1f-48e9-9634-4cdbf8d74168 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:19.720187 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e519f8-ce1f-48e9-9634-4cdbf8d74168 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:19.720699 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e519f8-ce1f-48e9-9634-4cdbf8d74168 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:19.722189 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e519f8-ce1f-48e9-9634-4cdbf8d74168 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:19.722704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e519f8-ce1f-48e9-9634-4cdbf8d74168 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:19.724391 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e519f8-ce1f-48e9-9634-4cdbf8d74168 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:19.737697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 333/659] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:19.738965 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e519f8-ce1f-48e9-9634-4cdbf8d74168 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:19.820388 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45062d1d-8cd0-495c-8711-041c3bbdec63 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:19.849203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e519f8-ce1f-48e9-9634-4cdbf8d74168 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:19.855984 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2adda-95cd-4f8a-885a-fbc2bda316dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:19.856648 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2adda-95cd-4f8a-885a-fbc2bda316dc None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:19.857228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2adda-95cd-4f8a-885a-fbc2bda316dc None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:19.857228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2adda-95cd-4f8a-885a-fbc2bda316dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:19.894130 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54490c75-6506-4b81-8a36-68d5947d4ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:19.895285 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54490c75-6506-4b81-8a36-68d5947d4ddd None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:19.895776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54490c75-6506-4b81-8a36-68d5947d4ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:19.896274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54490c75-6506-4b81-8a36-68d5947d4ddd None admin] PATH_INFO: `/v3/users/glance` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:19.907104 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54490c75-6506-4b81-8a36-68d5947d4ddd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:19.909491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54490c75-6506-4b81-8a36-68d5947d4ddd None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:19.926545 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2adda-95cd-4f8a-885a-fbc2bda316dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:19.928044 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2adda-95cd-4f8a-885a-fbc2bda316dc None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:19.929218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-54490c75-6506-4b81-8a36-68d5947d4ddd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 10 19:58:19.929218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:19.929218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:19.929218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:19.929218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:19.929218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:19.929218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:19.929218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:19.929218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:19.929218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:19.929218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:19.929218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:19.929218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:19.929218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:19.929218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:19.929218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:19.930180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:19.930180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:19.930180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:19.930180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:19.930180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:19.930180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:19.930180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:19.930180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:19.930180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:19.930180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:19.930180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:19.930180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:19.930180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:19.930180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:19.930180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:19.930180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:19.930180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:19.930180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:19.931170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:19.931170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:19.931170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:19.931170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:19.931170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:19.931170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:19.931170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:19.931170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:19.931170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 10 19:58:19.931170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:19.934146 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 334/660] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:58:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:19.935200 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54490c75-6506-4b81-8a36-68d5947d4ddd None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:20.031222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 327/661] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:58:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:58:20.032540 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864fee7-65aa-4e03-bf79-9e0fbf461459 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:20.032829 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864fee7-65aa-4e03-bf79-9e0fbf461459 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:20.033104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864fee7-65aa-4e03-bf79-9e0fbf461459 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:20.035096 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 328/662] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:20.035632 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864fee7-65aa-4e03-bf79-9e0fbf461459 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:20.054934 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54490c75-6506-4b81-8a36-68d5947d4ddd None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:20.088846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162e92c-62b4-4858-8855-9fa9f9a83fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:20.089959 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162e92c-62b4-4858-8855-9fa9f9a83fdb None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:20.090484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162e92c-62b4-4858-8855-9fa9f9a83fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:20.090947 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162e92c-62b4-4858-8855-9fa9f9a83fdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:20.098363 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162e92c-62b4-4858-8855-9fa9f9a83fdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:20.099977 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162e92c-62b4-4858-8855-9fa9f9a83fdb None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:20.114149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-8162e92c-62b4-4858-8855-9fa9f9a83fdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:20.114149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:20.114149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:20.114149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:20.114149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:20.114149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:20.114149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:20.114149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:20.114149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:20.114149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:20.114149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:20.114149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:20.114149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:20.114149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:20.114149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:20.115377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:20.115377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:20.115377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:20.115377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:20.115377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:20.115377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:20.115377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:20.115377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:20.115377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:20.115377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:20.115377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:20.115377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:20.115377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:20.115377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:20.115377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:20.115377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:20.115377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:20.116484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:20.116484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:20.116484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:20.116484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:20.116484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:20.116484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:20.116484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:20.116484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:20.118240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 335/663] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:20.119153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162e92c-62b4-4858-8855-9fa9f9a83fdb None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:20.137154 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864fee7-65aa-4e03-bf79-9e0fbf461459 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:20.173039 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a5632-52a3-414e-bc02-58aea7d20ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:20.173822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a5632-52a3-414e-bc02-58aea7d20ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:20.174262 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a5632-52a3-414e-bc02-58aea7d20ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:20.174606 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a5632-52a3-414e-bc02-58aea7d20ae8 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:20.175507 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a5632-52a3-414e-bc02-58aea7d20ae8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:20.175876 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a5632-52a3-414e-bc02-58aea7d20ae8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:20.177372 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a5632-52a3-414e-bc02-58aea7d20ae8 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:20.195725 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 329/664] 10.222.0.65 () {56 vars in 1137 bytes} [Tue May 10 19:58:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:20.197187 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fa9bb-1a02-46a4-a4ef-1990fbf01ed3 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:20.197536 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fa9bb-1a02-46a4-a4ef-1990fbf01ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:20.197877 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fa9bb-1a02-46a4-a4ef-1990fbf01ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:20.214621 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-3a0fa9bb-1a02-46a4-a4ef-1990fbf01ed3 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:20.215893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-3a0fa9bb-1a02-46a4-a4ef-1990fbf01ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:20.223964 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162e92c-62b4-4858-8855-9fa9f9a83fdb None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:20.257745 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089720fb-0d03-4b9c-9f07-ca0197e93f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:20.261529 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089720fb-0d03-4b9c-9f07-ca0197e93f34 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:20.261529 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089720fb-0d03-4b9c-9f07-ca0197e93f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:20.261529 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089720fb-0d03-4b9c-9f07-ca0197e93f34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:20.269890 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089720fb-0d03-4b9c-9f07-ca0197e93f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:20.271443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089720fb-0d03-4b9c-9f07-ca0197e93f34 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:20.290135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 330/665] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:20.290135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fa9bb-1a02-46a4-a4ef-1990fbf01ed3 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:20.301355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-089720fb-0d03-4b9c-9f07-ca0197e93f34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:20.301355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:20.301355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:20.301355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:20.301355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:20.301355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:20.301355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:20.301355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:20.301355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:20.301355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:20.301355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:20.301355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:20.301355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:20.301355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:20.301355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:20.302304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:20.302304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:20.302304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:20.302304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:20.302304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:20.302304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:20.302304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:20.302304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:20.302304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:20.302304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:20.302304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:20.302304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:20.302304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:20.302304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:20.302304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:20.302304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:20.302304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:20.303203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:20.303203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:20.303203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:20.303203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:20.303908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 336/666] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:20.305001 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089720fb-0d03-4b9c-9f07-ca0197e93f34 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:20.376817 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fa9bb-1a02-46a4-a4ef-1990fbf01ed3 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:20.412000 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089720fb-0d03-4b9c-9f07-ca0197e93f34 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:20.413022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cca6b-d504-40f3-bb2b-5c2bc96104e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:20.413683 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cca6b-d504-40f3-bb2b-5c2bc96104e5 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:20.413966 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cca6b-d504-40f3-bb2b-5c2bc96104e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:20.414288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cca6b-d504-40f3-bb2b-5c2bc96104e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:20.415087 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cca6b-d504-40f3-bb2b-5c2bc96104e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:20.415396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cca6b-d504-40f3-bb2b-5c2bc96104e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:20.421029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cca6b-d504-40f3-bb2b-5c2bc96104e5 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:20.432282 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 331/667] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:20.437930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cca6b-d504-40f3-bb2b-5c2bc96104e5 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:20.452711 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44cc4a5-eb0b-4a1c-bf76-b359b37087b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:20.453357 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44cc4a5-eb0b-4a1c-bf76-b359b37087b2 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:20.453635 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44cc4a5-eb0b-4a1c-bf76-b359b37087b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:20.453907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44cc4a5-eb0b-4a1c-bf76-b359b37087b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:20.523305 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44cc4a5-eb0b-4a1c-bf76-b359b37087b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:20.524761 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44cc4a5-eb0b-4a1c-bf76-b359b37087b2 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:20.536675 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cca6b-d504-40f3-bb2b-5c2bc96104e5 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:20.570056 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf775d-8256-442f-a189-731085f7c767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:20.571140 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf775d-8256-442f-a189-731085f7c767 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:20.571426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf775d-8256-442f-a189-731085f7c767 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:20.571705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf775d-8256-442f-a189-731085f7c767 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:20.572432 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf775d-8256-442f-a189-731085f7c767 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:20.572740 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf775d-8256-442f-a189-731085f7c767 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:20.574140 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf775d-8256-442f-a189-731085f7c767 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:20.586437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 332/668] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:20.587483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf775d-8256-442f-a189-731085f7c767 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:20.634434 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 337/669] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:58:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 19:58:20.635251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44cc4a5-eb0b-4a1c-bf76-b359b37087b2 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:20.686344 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf775d-8256-442f-a189-731085f7c767 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:20.718951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb60a5d-5046-42fc-a660-08ec1be824d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:20.719731 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb60a5d-5046-42fc-a660-08ec1be824d2 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:20.720009 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb60a5d-5046-42fc-a660-08ec1be824d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:20.720280 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb60a5d-5046-42fc-a660-08ec1be824d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:20.731225 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb60a5d-5046-42fc-a660-08ec1be824d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:20.732847 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44cc4a5-eb0b-4a1c-bf76-b359b37087b2 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:20.739799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-eeb60a5d-5046-42fc-a660-08ec1be824d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:58:20.739799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:20.739799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:20.739799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:20.739799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:20.739799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:20.739799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:20.739799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:20.739799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:20.739799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:20.739799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:20.739799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:20.739799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:20.739799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:20.739799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:20.739799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:20.740849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:20.740849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:20.740849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:20.740849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:20.740849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:20.740849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:20.740849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:20.740849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:20.740849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:20.740849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:20.740849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:20.740849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:20.740849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:20.740849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:20.740849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:20.740849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:20.741690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:20.741690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:20.741690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:20.741690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:20.741690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:20.741690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:20.741690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:20.741690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:20.741690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:20.741690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:20.741690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:58:20.741690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:20.741690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 333/670] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:58:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:20.742452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb60a5d-5046-42fc-a660-08ec1be824d2 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:20.778038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0e0a5-f466-43fa-b3a8-1ea106df159f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:20.778773 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0e0a5-f466-43fa-b3a8-1ea106df159f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:20.779058 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0e0a5-f466-43fa-b3a8-1ea106df159f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:20.779332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0e0a5-f466-43fa-b3a8-1ea106df159f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:20.780280 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0e0a5-f466-43fa-b3a8-1ea106df159f None admin] RBAC: Adding query filter params (role.id=55528a93cfa64162b2f9574fb5cdde79, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=0e366ec240e5464d8c5e92f807123c23) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:20.780591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0e0a5-f466-43fa-b3a8-1ea106df159f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:20.782028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0e0a5-f466-43fa-b3a8-1ea106df159f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:20.787896 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 338/671] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:58:20 2022] GET /identity/v3/role_assignments?user.id=0e366ec240e5464d8c5e92f807123c23&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=55528a93cfa64162b2f9574fb5cdde79 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:20.788665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0e0a5-f466-43fa-b3a8-1ea106df159f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:20.874050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb60a5d-5046-42fc-a660-08ec1be824d2 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:20.898739 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0e0a5-f466-43fa-b3a8-1ea106df159f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:20.912053 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789708ac-772e-4520-a519-4bd5f7b43b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:20.913116 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789708ac-772e-4520-a519-4bd5f7b43b8a None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:20.913585 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789708ac-772e-4520-a519-4bd5f7b43b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:20.914008 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789708ac-772e-4520-a519-4bd5f7b43b8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:20.921851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789708ac-772e-4520-a519-4bd5f7b43b8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:20.923636 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789708ac-772e-4520-a519-4bd5f7b43b8a None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:20.935112 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d88d0-3ec4-4d06-8dbb-312e89061c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:20.935912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d88d0-3ec4-4d06-8dbb-312e89061c52 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:20.936190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d88d0-3ec4-4d06-8dbb-312e89061c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:20.936508 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d88d0-3ec4-4d06-8dbb-312e89061c52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:20.940978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-789708ac-772e-4520-a519-4bd5f7b43b8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:20.940978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:20.940978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:20.940978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:20.940978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:20.940978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:20.940978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:20.940978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:20.940978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:20.940978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:20.940978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:20.940978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:20.940978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:20.940978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:20.940978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:20.942103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:20.942103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:20.942103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:20.942103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:20.942103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:20.942103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:20.942103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:20.942103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:20.942103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:20.942103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:20.942103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:20.942103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:20.942103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:20.942103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:20.942103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:20.942103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:20.942103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:20.943157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:20.943157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:20.943157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:20.943157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:20.943157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:20.943157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:20.943157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:20.943157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:20.943157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 334/672] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:20.943157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789708ac-772e-4520-a519-4bd5f7b43b8a None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:20.952825 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d88d0-3ec4-4d06-8dbb-312e89061c52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:20.954429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d88d0-3ec4-4d06-8dbb-312e89061c52 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:20.973355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-2e0d88d0-3ec4-4d06-8dbb-312e89061c52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:20.973355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:20.973355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:20.973355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:20.973355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:20.973355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:20.973355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:20.973355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:20.973355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:20.973355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:20.973355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:20.973355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:20.973355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:20.973355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:20.973355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:20.974531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:20.974531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:20.974531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:20.974531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:20.974531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:20.974531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:20.974531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:20.974531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:20.974531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:20.974531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:20.974531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:20.974531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:20.974531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:20.974531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:20.974531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:20.974531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:20.974531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:20.975583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:20.975583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:20.975583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:20.975583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:20.975583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 339/673] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:20.976276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b610f-ad5a-43cb-a2b0-fed2951806c9 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:20.976558 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b610f-ad5a-43cb-a2b0-fed2951806c9 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:20.976830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b610f-ad5a-43cb-a2b0-fed2951806c9 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:20.978280 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 340/674] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:20.982667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b610f-ad5a-43cb-a2b0-fed2951806c9 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:21.056278 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789708ac-772e-4520-a519-4bd5f7b43b8a None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:21.095245 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b610f-ad5a-43cb-a2b0-fed2951806c9 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:21.100469 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecc59f-7c4e-4814-b414-ed7bce0919c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:21.101137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecc59f-7c4e-4814-b414-ed7bce0919c4 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:21.101422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecc59f-7c4e-4814-b414-ed7bce0919c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:21.101697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecc59f-7c4e-4814-b414-ed7bce0919c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:21.102481 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecc59f-7c4e-4814-b414-ed7bce0919c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:21.102789 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecc59f-7c4e-4814-b414-ed7bce0919c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:21.104168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecc59f-7c4e-4814-b414-ed7bce0919c4 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:21.113185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 335/675] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:58:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:21.114135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecc59f-7c4e-4814-b414-ed7bce0919c4 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:21.134718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e1518-d620-4418-b51e-1e448976b73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:21.135442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e1518-d620-4418-b51e-1e448976b73a None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:21.135764 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e1518-d620-4418-b51e-1e448976b73a None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:21.136043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e1518-d620-4418-b51e-1e448976b73a None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:21.136836 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e1518-d620-4418-b51e-1e448976b73a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:21.137147 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e1518-d620-4418-b51e-1e448976b73a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:21.138715 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e1518-d620-4418-b51e-1e448976b73a None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:21.154805 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 341/676] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:21.154805 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757610e3-da5e-435f-8246-430d612250ae None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:21.154805 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757610e3-da5e-435f-8246-430d612250ae None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:21.154805 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757610e3-da5e-435f-8246-430d612250ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:21.168796 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-757610e3-da5e-435f-8246-430d612250ae None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:21.169945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-757610e3-da5e-435f-8246-430d612250ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:21.214968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecc59f-7c4e-4814-b414-ed7bce0919c4 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:21.246109 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 342/677] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:21.246109 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757610e3-da5e-435f-8246-430d612250ae None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:21.261733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f0d65-52e3-4af8-a008-dde1656d45bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:21.262795 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f0d65-52e3-4af8-a008-dde1656d45bc None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:21.263406 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f0d65-52e3-4af8-a008-dde1656d45bc None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:21.263786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f0d65-52e3-4af8-a008-dde1656d45bc None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:21.264750 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f0d65-52e3-4af8-a008-dde1656d45bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:21.265160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f0d65-52e3-4af8-a008-dde1656d45bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:21.266698 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f0d65-52e3-4af8-a008-dde1656d45bc None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:21.278942 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 336/678] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:21.279978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f0d65-52e3-4af8-a008-dde1656d45bc None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:21.340959 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757610e3-da5e-435f-8246-430d612250ae None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:21.376222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060ba74-a97f-4893-9a28-09f23e33b45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:21.377038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060ba74-a97f-4893-9a28-09f23e33b45f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:21.377168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f0d65-52e3-4af8-a008-dde1656d45bc None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:21.377333 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060ba74-a97f-4893-9a28-09f23e33b45f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:21.377587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060ba74-a97f-4893-9a28-09f23e33b45f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:21.388249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060ba74-a97f-4893-9a28-09f23e33b45f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:21.389700 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060ba74-a97f-4893-9a28-09f23e33b45f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:21.406622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-a060ba74-a97f-4893-9a28-09f23e33b45f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:21.406622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:21.406622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:21.406622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:21.406622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:21.406622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:21.406622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:21.406622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:21.406622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:21.406622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:21.406622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:21.406622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:21.406622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:21.406622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:21.406622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:21.407563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:21.407563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:21.407563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:21.407563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:21.407563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:21.407563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:21.407563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:21.407563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:21.407563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:21.407563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:21.407563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:21.407563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:21.407563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:21.407563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:21.407563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:21.407563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:21.407563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:21.409092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:21.409092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:21.409092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:21.409092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:21.409092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 343/679] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:21.409092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060ba74-a97f-4893-9a28-09f23e33b45f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:21.415950 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758bad67-12f2-4eb2-96b4-7a81fdf08176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:21.416604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758bad67-12f2-4eb2-96b4-7a81fdf08176 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:21.416886 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758bad67-12f2-4eb2-96b4-7a81fdf08176 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:21.417167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758bad67-12f2-4eb2-96b4-7a81fdf08176 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:21.418014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758bad67-12f2-4eb2-96b4-7a81fdf08176 None admin] RBAC: Adding query filter params (role.id=55528a93cfa64162b2f9574fb5cdde79, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=a6aabd1571d04bcfae842626fef5ed6d) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:21.418354 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758bad67-12f2-4eb2-96b4-7a81fdf08176 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:21.419777 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758bad67-12f2-4eb2-96b4-7a81fdf08176 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:21.426770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 337/680] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:58:21 2022] GET /identity/v3/role_assignments?user.id=a6aabd1571d04bcfae842626fef5ed6d&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=55528a93cfa64162b2f9574fb5cdde79 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:21.427710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758bad67-12f2-4eb2-96b4-7a81fdf08176 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:21.546280 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060ba74-a97f-4893-9a28-09f23e33b45f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:21.553284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758bad67-12f2-4eb2-96b4-7a81fdf08176 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:21.590890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739ac6d-862f-447e-9f61-6ef88cd28c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:21.591935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4b635-bc58-45cf-a623-7a93bc19a0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:21.592277 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739ac6d-862f-447e-9f61-6ef88cd28c1a None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:21.592755 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739ac6d-862f-447e-9f61-6ef88cd28c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:21.592868 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4b635-bc58-45cf-a623-7a93bc19a0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:21.593198 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4b635-bc58-45cf-a623-7a93bc19a0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:21.593485 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739ac6d-862f-447e-9f61-6ef88cd28c1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:21.593595 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4b635-bc58-45cf-a623-7a93bc19a0f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:21.602449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1739ac6d-862f-447e-9f61-6ef88cd28c1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:21.604398 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1739ac6d-862f-447e-9f61-6ef88cd28c1a None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:21.607103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb4b635-bc58-45cf-a623-7a93bc19a0f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:21.616137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-8cb4b635-bc58-45cf-a623-7a93bc19a0f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:21.616137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:21.616137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:21.616137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:21.616137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:21.616137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:21.616137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:21.616137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:21.616137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:21.616137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:21.616137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:21.616137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:21.616137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:21.616137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:21.616137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:21.616137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:21.617174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:21.617174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:21.617174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:21.617174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:21.617174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:21.617174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:21.617174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:21.617174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:21.617174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:21.617174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:21.617174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:21.617174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:21.617174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:21.617174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:21.617174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:21.617174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:21.618123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:21.618123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:21.618123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:21.618123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:21.618123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:21.618123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:21.618123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:21.618123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:21.618123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:21.618123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:21.618123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:21.618123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:21.618123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 344/681] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:21.618827 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4b635-bc58-45cf-a623-7a93bc19a0f8 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:21.623961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-1739ac6d-862f-447e-9f61-6ef88cd28c1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:21.623961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:21.623961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:21.623961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:21.623961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:21.623961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:21.623961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:21.623961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:21.623961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:21.623961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:21.623961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:21.623961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:21.623961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:21.623961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:21.623961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:21.624918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:21.624918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:21.624918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:21.624918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:21.624918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:21.624918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:21.624918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:21.624918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:21.624918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:21.624918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:21.624918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:21.624918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:21.624918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:21.624918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:21.624918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:21.624918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:21.624918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:21.625875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:21.625875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:21.625875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:21.625875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:21.625875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:21.625875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:21.625875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:21.625875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:21.628532 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 338/682] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:21.629570 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739ac6d-862f-447e-9f61-6ef88cd28c1a None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:21.761544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4b635-bc58-45cf-a623-7a93bc19a0f8 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:21.764782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739ac6d-862f-447e-9f61-6ef88cd28c1a None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:21.801825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82619e67-c2b4-497f-a3b1-74c50763149e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:21.802412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82619e67-c2b4-497f-a3b1-74c50763149e None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:21.802730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82619e67-c2b4-497f-a3b1-74c50763149e None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:21.802975 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82619e67-c2b4-497f-a3b1-74c50763149e None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:21.803839 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82619e67-c2b4-497f-a3b1-74c50763149e None admin] RBAC: Adding query filter params (name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:21.804178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82619e67-c2b4-497f-a3b1-74c50763149e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:21.805662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82619e67-c2b4-497f-a3b1-74c50763149e None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:21.808895 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9dffc-530b-487b-bb8a-d1f2ef8d402d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:21.809785 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9dffc-530b-487b-bb8a-d1f2ef8d402d None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:21.810309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9dffc-530b-487b-bb8a-d1f2ef8d402d None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:21.811245 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9dffc-530b-487b-bb8a-d1f2ef8d402d None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:21.812490 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9dffc-530b-487b-bb8a-d1f2ef8d402d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:21.812994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9dffc-530b-487b-bb8a-d1f2ef8d402d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:21.814306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 339/683] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 19:58:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:21.814804 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9dffc-530b-487b-bb8a-d1f2ef8d402d None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:21.815212 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82619e67-c2b4-497f-a3b1-74c50763149e None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:21.830650 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 345/684] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:21.832892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9dffc-530b-487b-bb8a-d1f2ef8d402d None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:21.917751 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82619e67-c2b4-497f-a3b1-74c50763149e None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:21.933324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9dffc-530b-487b-bb8a-d1f2ef8d402d None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:21.952506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c535a44-53cc-4847-825d-2707605228a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:21.953495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c535a44-53cc-4847-825d-2707605228a2 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:21.953972 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c535a44-53cc-4847-825d-2707605228a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:21.954466 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c535a44-53cc-4847-825d-2707605228a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:21.956116 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c535a44-53cc-4847-825d-2707605228a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:21.956618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c535a44-53cc-4847-825d-2707605228a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:21.958267 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c535a44-53cc-4847-825d-2707605228a2 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:21.969461 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159f568-63ed-465b-a40c-e8279c8e6037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:21.970320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159f568-63ed-465b-a40c-e8279c8e6037 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:21.970660 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159f568-63ed-465b-a40c-e8279c8e6037 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:21.970938 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159f568-63ed-465b-a40c-e8279c8e6037 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:21.973581 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 340/685] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:21.974764 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c535a44-53cc-4847-825d-2707605228a2 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:21.988481 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159f568-63ed-465b-a40c-e8279c8e6037 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:21.990330 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159f568-63ed-465b-a40c-e8279c8e6037 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:22.008735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-c159f568-63ed-465b-a40c-e8279c8e6037 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:22.008735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:22.008735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:22.008735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:22.008735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:22.008735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:22.008735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:22.008735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:22.008735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:22.008735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:22.008735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:22.008735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:22.008735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:22.008735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:22.008735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:22.009665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:22.009665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:22.009665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:22.009665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:22.009665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:22.009665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:22.009665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:22.009665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:22.009665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:22.009665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:22.009665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:22.009665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:22.009665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:22.009665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:22.009665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:22.009665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:22.009665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:22.010586 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:22.010586 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:22.010586 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:22.010586 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:22.010586 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 346/686] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:22.011400 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159f568-63ed-465b-a40c-e8279c8e6037 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:22.078703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c535a44-53cc-4847-825d-2707605228a2 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:22.111916 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159f568-63ed-465b-a40c-e8279c8e6037 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:22.113647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808e790-2881-4971-ac7b-b4b831629c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:22.114985 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808e790-2881-4971-ac7b-b4b831629c79 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:22.115485 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808e790-2881-4971-ac7b-b4b831629c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:22.115963 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808e790-2881-4971-ac7b-b4b831629c79 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:22.123526 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808e790-2881-4971-ac7b-b4b831629c79 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:22.125217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808e790-2881-4971-ac7b-b4b831629c79 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:22.140374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-a808e790-2881-4971-ac7b-b4b831629c79 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 19:58:22.140374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:22.140374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:22.140374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:22.140374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:22.140374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:22.140374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:22.140374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:22.140374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:22.140374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:22.140374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:22.140374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:22.140374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:22.140374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:22.140374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:22.140374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:22.141342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:22.141342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:22.141342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:22.141342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:22.141342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:22.141342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:22.141342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:22.141342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:22.141342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:22.141342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:22.141342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:22.141342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:22.141342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:22.141342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:22.141342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:22.141342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:22.141342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:22.141342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:22.142300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:22.142300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:22.142300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:22.142300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:22.142300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:22.142300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:22.142300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:22.142300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:22.142300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 19:58:22.142300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:22.145795 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 341/687] 10.222.0.65 () {56 vars in 1093 bytes} [Tue May 10 19:58:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:22.146825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808e790-2881-4971-ac7b-b4b831629c79 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:22.147719 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1b1d4-3bde-4a61-9e07-55514bc3fca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:22.148371 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1b1d4-3bde-4a61-9e07-55514bc3fca4 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:22.148651 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1b1d4-3bde-4a61-9e07-55514bc3fca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:22.148926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1b1d4-3bde-4a61-9e07-55514bc3fca4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:22.149768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde1b1d4-3bde-4a61-9e07-55514bc3fca4 None admin] RBAC: Adding query filter params (role.id=55528a93cfa64162b2f9574fb5cdde79, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=67a252cde71a455fb1dc4bf9c47c0490) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:22.150197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde1b1d4-3bde-4a61-9e07-55514bc3fca4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:22.151685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde1b1d4-3bde-4a61-9e07-55514bc3fca4 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:22.162225 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 347/688] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:58:22 2022] GET /identity/v3/role_assignments?user.id=67a252cde71a455fb1dc4bf9c47c0490&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=55528a93cfa64162b2f9574fb5cdde79 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:22.163165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1b1d4-3bde-4a61-9e07-55514bc3fca4 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:22.251573 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808e790-2881-4971-ac7b-b4b831629c79 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:22.287743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412f2c7-5fa9-4b70-8551-782ce3c57070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:22.288773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412f2c7-5fa9-4b70-8551-782ce3c57070 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:22.289251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412f2c7-5fa9-4b70-8551-782ce3c57070 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:22.289748 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412f2c7-5fa9-4b70-8551-782ce3c57070 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:22.290782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412f2c7-5fa9-4b70-8551-782ce3c57070 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:22.291287 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412f2c7-5fa9-4b70-8551-782ce3c57070 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:22.293089 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412f2c7-5fa9-4b70-8551-782ce3c57070 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:22.306013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 342/689] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:22.311890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412f2c7-5fa9-4b70-8551-782ce3c57070 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:22.349177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1b1d4-3bde-4a61-9e07-55514bc3fca4 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:22.428907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412f2c7-5fa9-4b70-8551-782ce3c57070 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:22.442848 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005391ea-4ad0-4945-9cdb-4d63f6c0dd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:22.443535 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005391ea-4ad0-4945-9cdb-4d63f6c0dd4b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:22.443818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005391ea-4ad0-4945-9cdb-4d63f6c0dd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:22.444092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005391ea-4ad0-4945-9cdb-4d63f6c0dd4b None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:22.445111 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005391ea-4ad0-4945-9cdb-4d63f6c0dd4b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:22.445429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005391ea-4ad0-4945-9cdb-4d63f6c0dd4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:22.447124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005391ea-4ad0-4945-9cdb-4d63f6c0dd4b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:22.469434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55806d24-385c-4d38-8f53-c6e5062016e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:22.470789 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55806d24-385c-4d38-8f53-c6e5062016e1 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:22.471273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55806d24-385c-4d38-8f53-c6e5062016e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:22.471744 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55806d24-385c-4d38-8f53-c6e5062016e1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:22.482116 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55806d24-385c-4d38-8f53-c6e5062016e1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:22.483965 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55806d24-385c-4d38-8f53-c6e5062016e1 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:22.490849 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 348/690] 10.222.0.65 () {56 vars in 1133 bytes} [Tue May 10 19:58:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:22.495953 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005391ea-4ad0-4945-9cdb-4d63f6c0dd4b None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:22.501474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-55806d24-385c-4d38-8f53-c6e5062016e1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 10 19:58:22.501474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:22.501474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:22.501474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:22.501474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:22.501474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:22.501474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:22.501474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:22.501474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:22.501474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:22.501474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:22.501474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:22.501474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:22.501474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:22.501474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:22.501474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:22.503097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:22.503097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:22.503097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:22.503097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:22.503097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:22.503097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:22.503097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:22.503097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:22.503097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:22.503097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:22.503097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:22.503097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:22.503097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:22.503097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:22.503097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:22.503097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:22.503097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:22.503097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:22.504027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:22.504027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:22.504027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:22.504027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:22.504027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:22.504027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:22.504027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:22.504027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:22.504027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 10 19:58:22.504027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:22.506792 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 343/691] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:58:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:22.512528 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55806d24-385c-4d38-8f53-c6e5062016e1 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:22.626316 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005391ea-4ad0-4945-9cdb-4d63f6c0dd4b None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:22.641825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55806d24-385c-4d38-8f53-c6e5062016e1 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:22.678256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36350de6-d670-489c-aa86-66d125991cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:22.679185 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36350de6-d670-489c-aa86-66d125991cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:22.679467 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36350de6-d670-489c-aa86-66d125991cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:22.679896 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36350de6-d670-489c-aa86-66d125991cf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:22.686217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46546f65-12a1-4108-bcf5-2ff7e48c22b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:22.686928 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46546f65-12a1-4108-bcf5-2ff7e48c22b3 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:22.687214 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46546f65-12a1-4108-bcf5-2ff7e48c22b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:22.687490 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46546f65-12a1-4108-bcf5-2ff7e48c22b3 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:22.688348 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46546f65-12a1-4108-bcf5-2ff7e48c22b3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:22.688656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46546f65-12a1-4108-bcf5-2ff7e48c22b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:22.690591 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46546f65-12a1-4108-bcf5-2ff7e48c22b3 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:22.716534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 344/692] 10.222.0.65 () {56 vars in 1135 bytes} [Tue May 10 19:58:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:22.722125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46546f65-12a1-4108-bcf5-2ff7e48c22b3 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:22.762976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36350de6-d670-489c-aa86-66d125991cf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:22.765083 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36350de6-d670-489c-aa86-66d125991cf7 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:22.847263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46546f65-12a1-4108-bcf5-2ff7e48c22b3 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:22.884227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47db75e-1a99-4a1c-97be-7001b0559754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:22.884802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47db75e-1a99-4a1c-97be-7001b0559754 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:22.885085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47db75e-1a99-4a1c-97be-7001b0559754 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:22.885360 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47db75e-1a99-4a1c-97be-7001b0559754 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:22.885647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 349/693] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:58:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 19:58:22.891149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36350de6-d670-489c-aa86-66d125991cf7 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:22.963846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47db75e-1a99-4a1c-97be-7001b0559754 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:22.965298 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47db75e-1a99-4a1c-97be-7001b0559754 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:23.065717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36350de6-d670-489c-aa86-66d125991cf7 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:23.071012 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 345/694] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:58:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 19:58:23.078819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47db75e-1a99-4a1c-97be-7001b0559754 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:23.135903 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14a507-3942-4fa4-b79e-0f94851769a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:23.136618 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14a507-3942-4fa4-b79e-0f94851769a4 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:23.136900 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14a507-3942-4fa4-b79e-0f94851769a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:23.137175 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14a507-3942-4fa4-b79e-0f94851769a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:23.147384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14a507-3942-4fa4-b79e-0f94851769a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:23.148823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14a507-3942-4fa4-b79e-0f94851769a4 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:23.170422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-9a14a507-3942-4fa4-b79e-0f94851769a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:23.170422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:23.170422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:23.170422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:23.170422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:23.170422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:23.170422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:23.170422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:23.170422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:23.170422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:23.170422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:23.170422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:23.170422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:23.170422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:23.170422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:23.171369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:23.171369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:23.171369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:23.171369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:23.171369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:23.171369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:23.171369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:23.171369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:23.171369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:23.171369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:23.171369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:23.171369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:23.171369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:23.171369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:23.171369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:23.171369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:23.171369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:23.172271 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:23.172271 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:23.172271 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:23.172271 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:23.172932 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 350/695] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:23.174443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac53f4-b448-4fb4-9c2b-56a1a0802ace None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:23.174734 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac53f4-b448-4fb4-9c2b-56a1a0802ace None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:23.175010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac53f4-b448-4fb4-9c2b-56a1a0802ace None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:23.178170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 351/696] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:23.178948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac53f4-b448-4fb4-9c2b-56a1a0802ace None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:23.203847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47db75e-1a99-4a1c-97be-7001b0559754 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:23.237877 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae574e-92e8-4d89-8b2b-265d89b2a23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:23.238705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae574e-92e8-4d89-8b2b-265d89b2a23b None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:23.239159 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae574e-92e8-4d89-8b2b-265d89b2a23b None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:23.239717 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae574e-92e8-4d89-8b2b-265d89b2a23b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:23.250056 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae574e-92e8-4d89-8b2b-265d89b2a23b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:23.251585 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae574e-92e8-4d89-8b2b-265d89b2a23b None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:23.269651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-75ae574e-92e8-4d89-8b2b-265d89b2a23b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:23.269651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:23.269651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:23.269651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:23.269651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:23.269651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:23.269651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:23.269651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:23.269651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:23.269651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:23.269651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:23.269651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:23.269651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:23.269651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:23.269651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:23.270790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:23.270790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:23.270790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:23.270790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:23.270790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:23.270790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:23.270790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:23.270790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:23.270790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:23.270790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:23.270790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:23.270790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:23.270790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:23.270790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:23.270790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:23.270790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:23.270790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:23.271705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:23.271705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:23.271705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:23.271705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:23.271705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 346/697] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:23.272552 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11400660-e9e9-41ab-95fc-3223f8fdc9fa None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:23.272841 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11400660-e9e9-41ab-95fc-3223f8fdc9fa None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:23.273119 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11400660-e9e9-41ab-95fc-3223f8fdc9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:23.290927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-11400660-e9e9-41ab-95fc-3223f8fdc9fa None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:23.292103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-11400660-e9e9-41ab-95fc-3223f8fdc9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:23.302877 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac53f4-b448-4fb4-9c2b-56a1a0802ace None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:23.350236 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493f9a6-2b10-4534-80b8-a7cff2fdf665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:23.350895 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493f9a6-2b10-4534-80b8-a7cff2fdf665 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:23.351179 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493f9a6-2b10-4534-80b8-a7cff2fdf665 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:23.351452 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493f9a6-2b10-4534-80b8-a7cff2fdf665 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:23.352176 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493f9a6-2b10-4534-80b8-a7cff2fdf665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:23.352480 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493f9a6-2b10-4534-80b8-a7cff2fdf665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:23.353858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493f9a6-2b10-4534-80b8-a7cff2fdf665 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:23.366627 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 352/699] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:23.367515 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493f9a6-2b10-4534-80b8-a7cff2fdf665 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:23.370210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 347/699] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:23.372158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11400660-e9e9-41ab-95fc-3223f8fdc9fa None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:23.484147 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493f9a6-2b10-4534-80b8-a7cff2fdf665 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:23.502505 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11400660-e9e9-41ab-95fc-3223f8fdc9fa None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:23.520119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a0d3c-0ed4-4bbd-8487-e183ea039262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:23.520929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a0d3c-0ed4-4bbd-8487-e183ea039262 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:23.521556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a0d3c-0ed4-4bbd-8487-e183ea039262 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:23.521556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a0d3c-0ed4-4bbd-8487-e183ea039262 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:23.522597 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a0d3c-0ed4-4bbd-8487-e183ea039262 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:23.522941 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a0d3c-0ed4-4bbd-8487-e183ea039262 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:23.524443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a0d3c-0ed4-4bbd-8487-e183ea039262 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:23.537262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 353/700] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:23.538321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a0d3c-0ed4-4bbd-8487-e183ea039262 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:23.545559 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db1def-0baf-4c51-9d5c-2f5bf330cb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:23.546606 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db1def-0baf-4c51-9d5c-2f5bf330cb33 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:23.547046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db1def-0baf-4c51-9d5c-2f5bf330cb33 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:23.547469 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db1def-0baf-4c51-9d5c-2f5bf330cb33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:23.557313 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db1def-0baf-4c51-9d5c-2f5bf330cb33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:23.559500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db1def-0baf-4c51-9d5c-2f5bf330cb33 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:23.577478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-e7db1def-0baf-4c51-9d5c-2f5bf330cb33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:23.577478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:23.577478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:23.577478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:23.577478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:23.577478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:23.577478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:23.577478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:23.577478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:23.577478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:23.577478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:23.577478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:23.577478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:23.577478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:23.577478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:23.578608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:23.578608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:23.578608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:23.578608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:23.578608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:23.578608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:23.578608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:23.578608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:23.578608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:23.578608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:23.578608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:23.578608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:23.578608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:23.578608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:23.578608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:23.578608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:23.578608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:23.579491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:23.579491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:23.579491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:23.579491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:23.579491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:23.579491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:23.579491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:23.579491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:23.581948 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 348/701] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:23.582939 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db1def-0baf-4c51-9d5c-2f5bf330cb33 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:23.650056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a0d3c-0ed4-4bbd-8487-e183ea039262 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:23.692189 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f08b2b-1166-4121-9d27-035a5ad19afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:23.693693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f08b2b-1166-4121-9d27-035a5ad19afb None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:23.694401 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f08b2b-1166-4121-9d27-035a5ad19afb None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:23.694851 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f08b2b-1166-4121-9d27-035a5ad19afb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:23.695825 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f08b2b-1166-4121-9d27-035a5ad19afb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:23.697968 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f08b2b-1166-4121-9d27-035a5ad19afb None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:23.706165 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db1def-0baf-4c51-9d5c-2f5bf330cb33 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:23.716643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-f0f08b2b-1166-4121-9d27-035a5ad19afb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 19:58:23.716643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:23.716643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:23.716643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:23.716643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:23.716643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:23.716643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:23.716643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:23.716643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 19:58:23.716643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:23.716643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:23.716643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:23.716643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:23.716643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:23.716643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:23.716643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:23.717852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:23.717852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:23.717852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:23.717852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:23.717852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:23.717852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:23.717852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:23.717852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:23.717852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:23.717852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:23.717852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:23.717852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:23.717852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:23.717852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:23.717852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:23.717852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:23.718927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:23.718927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:23.718927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:23.718927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:23.718927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:23.718927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:23.718927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:23.718927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:23.718927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:23.718927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:23.718927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:23.718927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:23.718927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:23.718927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 19:58:23.718927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:23.727675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 354/702] 10.222.0.65 () {56 vars in 1129 bytes} [Tue May 10 19:58:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:23.727675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f08b2b-1166-4121-9d27-035a5ad19afb None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:23.752124 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf20e9-9b3f-4947-b73b-262ffc6a0016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:23.752935 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf20e9-9b3f-4947-b73b-262ffc6a0016 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:23.753214 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf20e9-9b3f-4947-b73b-262ffc6a0016 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:23.753482 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf20e9-9b3f-4947-b73b-262ffc6a0016 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:23.763762 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf20e9-9b3f-4947-b73b-262ffc6a0016 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:23.765543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf20e9-9b3f-4947-b73b-262ffc6a0016 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:23.787136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-e8cf20e9-9b3f-4947-b73b-262ffc6a0016 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:23.787136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:23.787136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:23.787136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:23.787136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:23.787136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:23.787136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:23.787136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:23.787136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:23.787136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:23.787136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:23.787136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:23.787136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:23.787136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:23.787136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:23.788099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:23.788099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:23.788099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:23.788099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:23.788099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:23.788099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:23.788099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:23.788099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:23.788099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:23.788099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:23.788099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:23.788099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:23.788099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:23.788099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:23.788099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:23.788099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:23.788099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:23.788968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:23.788968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:23.788968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:23.788968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:23.788968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:23.788968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:23.788968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:23.788968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:23.788968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 349/703] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:23.789964 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf20e9-9b3f-4947-b73b-262ffc6a0016 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:23.863073 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f08b2b-1166-4121-9d27-035a5ad19afb None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:23.926578 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d6dda-3b76-4632-9210-e4e42fec3ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:23.928263 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d6dda-3b76-4632-9210-e4e42fec3ead None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:23.929350 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d6dda-3b76-4632-9210-e4e42fec3ead None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:23.929984 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d6dda-3b76-4632-9210-e4e42fec3ead None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:23.932020 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d6dda-3b76-4632-9210-e4e42fec3ead None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:23.932457 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d6dda-3b76-4632-9210-e4e42fec3ead None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:23.934246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d6dda-3b76-4632-9210-e4e42fec3ead None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:23.938967 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf20e9-9b3f-4947-b73b-262ffc6a0016 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:23.947106 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 355/704] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:23.947880 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d6dda-3b76-4632-9210-e4e42fec3ead None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:23.978337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed5a42-74fa-40aa-aed6-68226cb433d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:23.979002 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed5a42-74fa-40aa-aed6-68226cb433d1 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:23.979289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed5a42-74fa-40aa-aed6-68226cb433d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:23.979562 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed5a42-74fa-40aa-aed6-68226cb433d1 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:23.980266 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed5a42-74fa-40aa-aed6-68226cb433d1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:23.980579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed5a42-74fa-40aa-aed6-68226cb433d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:23.981956 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed5a42-74fa-40aa-aed6-68226cb433d1 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:23.991859 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 350/705] 10.222.0.65 () {56 vars in 1124 bytes} [Tue May 10 19:58:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:23.991859 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2734e3-2285-438c-bb26-1d2212751ff3 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:23.991859 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2734e3-2285-438c-bb26-1d2212751ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:23.991859 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2734e3-2285-438c-bb26-1d2212751ff3 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:23.994226 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 351/706] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:23.995492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2734e3-2285-438c-bb26-1d2212751ff3 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:24.054767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d6dda-3b76-4632-9210-e4e42fec3ead None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:24.088053 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc743d-1f13-443c-a944-00380a7ccf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:24.088732 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc743d-1f13-443c-a944-00380a7ccf64 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:24.089034 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc743d-1f13-443c-a944-00380a7ccf64 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:24.089304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc743d-1f13-443c-a944-00380a7ccf64 None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:24.090164 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc743d-1f13-443c-a944-00380a7ccf64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:24.090491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc743d-1f13-443c-a944-00380a7ccf64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:24.091918 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc743d-1f13-443c-a944-00380a7ccf64 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:24.104088 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 356/707] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:24.104453 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2734e3-2285-438c-bb26-1d2212751ff3 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:24.104883 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc743d-1f13-443c-a944-00380a7ccf64 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:24.144284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78b941-decd-4889-a3c1-47cc50bcf3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:24.144931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78b941-decd-4889-a3c1-47cc50bcf3df None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:24.145215 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78b941-decd-4889-a3c1-47cc50bcf3df None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:24.145491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78b941-decd-4889-a3c1-47cc50bcf3df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:24.146361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78b941-decd-4889-a3c1-47cc50bcf3df None admin] RBAC: Adding query filter params (role.id=5c87e804d0764309ba08ad6e06b23a87, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=ae922b2da4814d10bed1fa345700ac04) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:24.146675 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78b941-decd-4889-a3c1-47cc50bcf3df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:24.148141 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78b941-decd-4889-a3c1-47cc50bcf3df None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:24.153668 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 352/708] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:58:23 2022] GET /identity/v3/role_assignments?user.id=ae922b2da4814d10bed1fa345700ac04&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:24.154902 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e7658-86c5-4c42-89b0-874d65496b0e None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:24.155189 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e7658-86c5-4c42-89b0-874d65496b0e None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:24.155465 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e7658-86c5-4c42-89b0-874d65496b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:24.178812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-da6e7658-86c5-4c42-89b0-874d65496b0e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:24.180887 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-da6e7658-86c5-4c42-89b0-874d65496b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:24.222985 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc743d-1f13-443c-a944-00380a7ccf64 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:24.259832 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da497555-8176-4bed-ae13-49ddda3df279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:24.260492 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da497555-8176-4bed-ae13-49ddda3df279 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:24.260775 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da497555-8176-4bed-ae13-49ddda3df279 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:24.261050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da497555-8176-4bed-ae13-49ddda3df279 None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:24.261781 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da497555-8176-4bed-ae13-49ddda3df279 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:24.262112 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da497555-8176-4bed-ae13-49ddda3df279 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:24.263540 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da497555-8176-4bed-ae13-49ddda3df279 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:24.270681 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 357/709] 10.222.0.65 () {56 vars in 1124 bytes} [Tue May 10 19:58:24 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:24.271571 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da497555-8176-4bed-ae13-49ddda3df279 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:24.274862 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 353/710] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:24.326104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e7658-86c5-4c42-89b0-874d65496b0e None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:24.534435 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e7658-86c5-4c42-89b0-874d65496b0e None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:24.582541 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da497555-8176-4bed-ae13-49ddda3df279 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:24.590017 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cd2f1-f9fc-4e68-9d02-c33f57ff0c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:24.590982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cd2f1-f9fc-4e68-9d02-c33f57ff0c08 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:24.591368 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cd2f1-f9fc-4e68-9d02-c33f57ff0c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:24.591743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cd2f1-f9fc-4e68-9d02-c33f57ff0c08 None admin] PATH_INFO: `/v3/services/network` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:24.592508 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9cd2f1-f9fc-4e68-9d02-c33f57ff0c08 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:24.593992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9cd2f1-f9fc-4e68-9d02-c33f57ff0c08 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:24.608134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-ac9cd2f1-f9fc-4e68-9d02-c33f57ff0c08 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 10 19:58:24.608134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:24.608134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:24.608134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:24.608134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:24.608134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:24.608134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:24.608134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:24.608134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 10 19:58:24.608134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:24.608134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:24.608134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:24.608134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:24.608134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:24.608134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:24.608134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:24.608134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:24.609433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:24.609433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:24.609433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:24.609433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:24.609433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:24.609433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:24.609433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:24.609433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:24.609433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:24.609433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:24.609433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:24.609433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:24.609433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:24.609433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:24.609433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:24.609433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:24.610510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:24.610510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:24.610510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:24.610510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:24.610510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:24.610510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:24.610510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:24.610510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:24.610510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:24.610510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:24.610510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:24.610510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:24.610510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 10 19:58:24.610510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:24.613190 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 354/711] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:24 2022] GET /identity/v3/services/network => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:24.619323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cd2f1-f9fc-4e68-9d02-c33f57ff0c08 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:24.626451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed86eb2-95d8-4b0c-b4b6-530e0d71aa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:24.627907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed86eb2-95d8-4b0c-b4b6-530e0d71aa8d None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:24.628336 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed86eb2-95d8-4b0c-b4b6-530e0d71aa8d None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:24.628751 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed86eb2-95d8-4b0c-b4b6-530e0d71aa8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:24.629736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed86eb2-95d8-4b0c-b4b6-530e0d71aa8d None admin] RBAC: Adding query filter params (role.id=55528a93cfa64162b2f9574fb5cdde79, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=b4d10b258a53484da22950e2b1db2e41) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:24.630269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed86eb2-95d8-4b0c-b4b6-530e0d71aa8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:24.633238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed86eb2-95d8-4b0c-b4b6-530e0d71aa8d None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:24.640086 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 358/712] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:58:24 2022] GET /identity/v3/role_assignments?user.id=b4d10b258a53484da22950e2b1db2e41&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=55528a93cfa64162b2f9574fb5cdde79 => generated 608 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:24.735085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cd2f1-f9fc-4e68-9d02-c33f57ff0c08 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:24.778470 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3e947-6ae4-4579-8281-3816dd60679e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:24.779406 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3e947-6ae4-4579-8281-3816dd60679e None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:24.779690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3e947-6ae4-4579-8281-3816dd60679e None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:24.779966 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3e947-6ae4-4579-8281-3816dd60679e None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:24.780696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3e947-6ae4-4579-8281-3816dd60679e None admin] RBAC: Adding query filter params (name=network) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:24.781000 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3e947-6ae4-4579-8281-3816dd60679e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:24.787291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3e947-6ae4-4579-8281-3816dd60679e None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:24.792424 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 355/713] 10.222.0.65 () {56 vars in 1112 bytes} [Tue May 10 19:58:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:24.797916 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed86eb2-95d8-4b0c-b4b6-530e0d71aa8d None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:24.925123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed86eb2-95d8-4b0c-b4b6-530e0d71aa8d None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:24.959347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1ba83-5080-41ca-914a-b593df529a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:24.960254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1ba83-5080-41ca-914a-b593df529a66 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:24.960685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1ba83-5080-41ca-914a-b593df529a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:24.961125 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1ba83-5080-41ca-914a-b593df529a66 None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:24.961989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1ba83-5080-41ca-914a-b593df529a66 None admin] RBAC: Adding query filter params (type=network) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:24.965357 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1ba83-5080-41ca-914a-b593df529a66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:24.966973 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1ba83-5080-41ca-914a-b593df529a66 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:24.972799 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 359/714] 10.222.0.65 () {56 vars in 1112 bytes} [Tue May 10 19:58:24 2022] GET /identity/v3/services?type=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:25.265504 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d2375-e068-4aac-af30-62c92e0a46b4 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:25.266489 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d2375-e068-4aac-af30-62c92e0a46b4 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:25.269048 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d2375-e068-4aac-af30-62c92e0a46b4 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:25.269506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 356/715] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:25.275471 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15cfe1-4f9a-46ae-922a-8bc85bcd519f None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:25.276021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15cfe1-4f9a-46ae-922a-8bc85bcd519f None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:25.276522 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15cfe1-4f9a-46ae-922a-8bc85bcd519f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:25.294605 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-ad15cfe1-4f9a-46ae-922a-8bc85bcd519f None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:25.296156 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-ad15cfe1-4f9a-46ae-922a-8bc85bcd519f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:25.365034 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 360/716] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:25.389219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d2375-e068-4aac-af30-62c92e0a46b4 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:25.529780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d2375-e068-4aac-af30-62c92e0a46b4 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:25.574849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc8fcd-5c7e-480f-9d72-fd56abf16893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:25.575661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc8fcd-5c7e-480f-9d72-fd56abf16893 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:25.575951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc8fcd-5c7e-480f-9d72-fd56abf16893 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:25.576228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc8fcd-5c7e-480f-9d72-fd56abf16893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:25.589932 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc8fcd-5c7e-480f-9d72-fd56abf16893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:25.591749 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc8fcd-5c7e-480f-9d72-fd56abf16893 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:25.612713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-5cdc8fcd-5c7e-480f-9d72-fd56abf16893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:25.612713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:25.612713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:25.612713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:25.612713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:25.612713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:25.612713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:25.612713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:25.612713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:25.612713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:25.612713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:25.612713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:25.612713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:25.612713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:25.612713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:25.613647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:25.613647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:25.613647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:25.613647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:25.613647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:25.613647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:25.613647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:25.613647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:25.613647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:25.613647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:25.613647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:25.613647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:25.613647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:25.613647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:25.613647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:25.613647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:25.613647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:25.614846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:25.614846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:25.614846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:25.614846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:25.617607 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 357/717] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:25.625232 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15cfe1-4f9a-46ae-922a-8bc85bcd519f None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:25.749583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15cfe1-4f9a-46ae-922a-8bc85bcd519f None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:25.784492 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2723d-7c89-46a4-a6fa-da3d6c474ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:25.785070 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2723d-7c89-46a4-a6fa-da3d6c474ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:25.785351 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2723d-7c89-46a4-a6fa-da3d6c474ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:25.785621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2723d-7c89-46a4-a6fa-da3d6c474ed3 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:25.786383 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2723d-7c89-46a4-a6fa-da3d6c474ed3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:25.786685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2723d-7c89-46a4-a6fa-da3d6c474ed3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:25.788042 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2723d-7c89-46a4-a6fa-da3d6c474ed3 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:25.799921 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 361/718] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:25.805298 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc8fcd-5c7e-480f-9d72-fd56abf16893 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:25.943201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc8fcd-5c7e-480f-9d72-fd56abf16893 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:25.991278 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f8368-2e6e-4c62-9645-c64e4b8aa2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:25.991950 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f8368-2e6e-4c62-9645-c64e4b8aa2b9 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:25.992240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f8368-2e6e-4c62-9645-c64e4b8aa2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:25.992513 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f8368-2e6e-4c62-9645-c64e4b8aa2b9 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:25.993239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f8368-2e6e-4c62-9645-c64e4b8aa2b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:25.994696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f8368-2e6e-4c62-9645-c64e4b8aa2b9 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:26.093636 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 358/719] 10.222.0.65 () {60 vars in 1132 bytes} [Tue May 10 19:58:25 2022] POST /identity/v3/users => generated 254 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:58:26.792322 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01feea37-3aef-46cb-baee-395d71e7d569 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:26.793020 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01feea37-3aef-46cb-baee-395d71e7d569 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:26.793517 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01feea37-3aef-46cb-baee-395d71e7d569 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:26.795424 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 362/720] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:26.801010 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a26980-249b-4a86-be1e-db66ed76ffbc None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:26.801886 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a26980-249b-4a86-be1e-db66ed76ffbc None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:26.802405 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a26980-249b-4a86-be1e-db66ed76ffbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:26.820546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-b0a26980-249b-4a86-be1e-db66ed76ffbc None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:26.822062 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b0a26980-249b-4a86-be1e-db66ed76ffbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:26.912446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 359/721] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:26.946657 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01feea37-3aef-46cb-baee-395d71e7d569 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:27.069279 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01feea37-3aef-46cb-baee-395d71e7d569 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:27.069581 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6256d-628b-4d1e-8720-dd62b57a2e15 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:27.069778 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6256d-628b-4d1e-8720-dd62b57a2e15 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:27.070057 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6256d-628b-4d1e-8720-dd62b57a2e15 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:27.071756 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 360/722] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:27.083935 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29165198-b214-4c25-a7b8-1e1f5d3b58f8 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:27.084238 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29165198-b214-4c25-a7b8-1e1f5d3b58f8 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:27.084502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29165198-b214-4c25-a7b8-1e1f5d3b58f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:27.105491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63144703-438e-4436-9a27-1c139ee930b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:27.106774 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63144703-438e-4436-9a27-1c139ee930b1 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:27.107270 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63144703-438e-4436-9a27-1c139ee930b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:27.107749 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63144703-438e-4436-9a27-1c139ee930b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:27.116498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-29165198-b214-4c25-a7b8-1e1f5d3b58f8 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:27.120480 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-29165198-b214-4c25-a7b8-1e1f5d3b58f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:27.133914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63144703-438e-4436-9a27-1c139ee930b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:27.141853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-63144703-438e-4436-9a27-1c139ee930b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:58:27.141853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:27.141853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:27.141853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:27.141853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:27.141853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:27.141853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:27.141853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:27.141853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:27.141853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:27.141853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:27.141853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:27.141853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:27.141853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:27.141853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:27.141853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:27.145503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:27.145503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:27.145503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:27.145503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:27.145503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:27.145503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:27.145503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:27.145503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:27.145503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:27.145503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:27.145503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:27.145503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:27.145503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:27.145503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:27.145503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:27.145503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:27.146963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:27.146963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:27.146963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:27.146963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:27.146963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:27.146963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:27.146963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:27.146963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:27.146963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:27.146963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:27.146963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:58:27.146963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:27.154535 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 363/723] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:58:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:27.167149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63144703-438e-4436-9a27-1c139ee930b1 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:27.224844 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 361/724] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 19:58:27.268235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29165198-b214-4c25-a7b8-1e1f5d3b58f8 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:27.349828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63144703-438e-4436-9a27-1c139ee930b1 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:27.378174 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29165198-b214-4c25-a7b8-1e1f5d3b58f8 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:27.416674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a11da-7d3f-45ee-b2b2-303b2acf3b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:27.420330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a11da-7d3f-45ee-b2b2-303b2acf3b68 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:27.420330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a11da-7d3f-45ee-b2b2-303b2acf3b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:27.420330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a11da-7d3f-45ee-b2b2-303b2acf3b68 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:27.420330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a11da-7d3f-45ee-b2b2-303b2acf3b68 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:27.420330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a11da-7d3f-45ee-b2b2-303b2acf3b68 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:27.438140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d586e92-5f19-4daf-96d5-ea52fe2b40e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:27.438140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d586e92-5f19-4daf-96d5-ea52fe2b40e8 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:27.438140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d586e92-5f19-4daf-96d5-ea52fe2b40e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:27.438140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d586e92-5f19-4daf-96d5-ea52fe2b40e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:27.447207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d586e92-5f19-4daf-96d5-ea52fe2b40e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:27.447207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d586e92-5f19-4daf-96d5-ea52fe2b40e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:27.447207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d586e92-5f19-4daf-96d5-ea52fe2b40e8 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:27.449465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 364/725] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:58:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:27.456499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d586e92-5f19-4daf-96d5-ea52fe2b40e8 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:27.527597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 362/726] 10.222.0.65 () {60 vars in 1141 bytes} [Tue May 10 19:58:27 2022] POST /identity/v3/services => generated 251 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 19:58:27.577229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d586e92-5f19-4daf-96d5-ea52fe2b40e8 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:27.648517 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edad20-955f-4f03-a63a-ea4cceeecdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:27.649248 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edad20-955f-4f03-a63a-ea4cceeecdbe None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:27.649581 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edad20-955f-4f03-a63a-ea4cceeecdbe None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:27.649910 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edad20-955f-4f03-a63a-ea4cceeecdbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:27.666885 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98edad20-955f-4f03-a63a-ea4cceeecdbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:27.668450 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98edad20-955f-4f03-a63a-ea4cceeecdbe None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:27.669970 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0a4aa-08bd-4f60-9326-e0ee32932a43 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:27.670281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0a4aa-08bd-4f60-9326-e0ee32932a43 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:27.671354 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0a4aa-08bd-4f60-9326-e0ee32932a43 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:27.672217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 363/727] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:27.693099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfb808-12e4-4b67-8838-e2a6ccf1c13a None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:27.693418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-98edad20-955f-4f03-a63a-ea4cceeecdbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:27.693418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:27.693418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:27.693418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:27.693418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:27.693418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:27.693418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:27.693418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:27.693418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:27.693418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:27.693418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:27.693418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:27.693418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:27.693418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:27.693418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:27.695625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:27.695625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:27.695625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:27.695625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:27.695625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:27.695625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:27.695625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:27.695625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:27.695625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:27.695625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:27.695625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:27.695625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:27.695625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:27.695625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:27.695625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:27.695625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:27.695625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:27.697835 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:27.697835 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:27.697835 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:27.697835 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:27.698432 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfb808-12e4-4b67-8838-e2a6ccf1c13a None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:27.698432 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfb808-12e4-4b67-8838-e2a6ccf1c13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:27.698723 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 365/728] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:27.700350 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edad20-955f-4f03-a63a-ea4cceeecdbe None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:27.717606 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-8fbfb808-12e4-4b67-8838-e2a6ccf1c13a None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:27.719211 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8fbfb808-12e4-4b67-8838-e2a6ccf1c13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:27.813701 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 364/729] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:58:27.815552 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a14d73-017c-45ac-aa5e-09b4de0c22e0 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:27.815835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a14d73-017c-45ac-aa5e-09b4de0c22e0 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:27.816173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a14d73-017c-45ac-aa5e-09b4de0c22e0 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:27.817629 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 365/730] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:27.822923 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4a0ab-8d5b-4df5-a39d-593716b2e0cd None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:27.823211 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4a0ab-8d5b-4df5-a39d-593716b2e0cd None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:27.823532 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4a0ab-8d5b-4df5-a39d-593716b2e0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:27.838773 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edad20-955f-4f03-a63a-ea4cceeecdbe None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:27.841361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-1dd4a0ab-8d5b-4df5-a39d-593716b2e0cd None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:27.841619 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1dd4a0ab-8d5b-4df5-a39d-593716b2e0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:27.888489 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bb390-ba52-4b81-8071-def3623c1375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:27.889206 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bb390-ba52-4b81-8071-def3623c1375 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:27.889437 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bb390-ba52-4b81-8071-def3623c1375 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:27.889713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bb390-ba52-4b81-8071-def3623c1375 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:27.890549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bb390-ba52-4b81-8071-def3623c1375 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:27.890994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bb390-ba52-4b81-8071-def3623c1375 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:27.892303 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bb390-ba52-4b81-8071-def3623c1375 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:27.904983 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 366/731] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:27.905888 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bb390-ba52-4b81-8071-def3623c1375 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:27.934107 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 366/732] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 19:58:27.934107 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4a0ab-8d5b-4df5-a39d-593716b2e0cd None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:28.036653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bb390-ba52-4b81-8071-def3623c1375 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:28.045167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4a0ab-8d5b-4df5-a39d-593716b2e0cd None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:28.086597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb30f0-7aa8-4380-8bed-67f51c96ef6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:28.086597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb30f0-7aa8-4380-8bed-67f51c96ef6a None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:28.086597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb30f0-7aa8-4380-8bed-67f51c96ef6a None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:28.086597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb30f0-7aa8-4380-8bed-67f51c96ef6a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:28.088929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4346f74-d3a7-4171-8705-a01ce8e448f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:28.089968 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4346f74-d3a7-4171-8705-a01ce8e448f3 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:28.090485 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4346f74-d3a7-4171-8705-a01ce8e448f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:28.090972 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4346f74-d3a7-4171-8705-a01ce8e448f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:28.092272 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb30f0-7aa8-4380-8bed-67f51c96ef6a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:28.093709 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb30f0-7aa8-4380-8bed-67f51c96ef6a None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:28.103421 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4346f74-d3a7-4171-8705-a01ce8e448f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:28.109666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-2ddb30f0-7aa8-4380-8bed-67f51c96ef6a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 19:58:28.109666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:28.109666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:28.109666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:28.109666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:28.109666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:28.109666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:28.109666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:28.109666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:28.109666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:28.109666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:28.109666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:28.109666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:28.109666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:28.109666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:28.109666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:28.110667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:28.110667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:28.110667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:28.110667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:28.110667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:28.110667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:28.110667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:28.110667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:28.110667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:28.110667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:28.110667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:28.110667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:28.110667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:28.110667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:28.110667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:28.110667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:28.110667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:28.110667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:28.113311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:28.113311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:28.113311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:28.113311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:28.113311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:28.113311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:28.113311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:28.113311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:28.113311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 19:58:28.113311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:28.113311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 367/733] 10.222.0.65 () {56 vars in 1093 bytes} [Tue May 10 19:58:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:28.113311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb30f0-7aa8-4380-8bed-67f51c96ef6a None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:28.120355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-b4346f74-d3a7-4171-8705-a01ce8e448f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:58:28.120355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:28.120355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:28.120355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:28.120355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:28.120355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:28.120355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:28.120355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:28.120355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:28.120355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:28.120355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:28.120355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:28.120355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:28.120355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:28.120355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:28.120355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:28.121288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:28.121288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:28.121288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:28.121288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:28.121288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:28.121288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:28.121288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:28.121288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:28.121288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:28.121288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:28.121288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:28.121288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:28.121288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:28.121288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:28.121288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:28.121288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:28.123521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:28.123521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:28.123521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:28.123521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:28.123521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:28.123521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:28.123521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:28.123521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:28.123521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:28.123521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:28.123521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:58:28.123521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:28.123521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 367/734] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:58:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:28.123521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4346f74-d3a7-4171-8705-a01ce8e448f3 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:28.245210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb30f0-7aa8-4380-8bed-67f51c96ef6a None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:28.254183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4346f74-d3a7-4171-8705-a01ce8e448f3 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:28.281878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdaa28-a376-4c54-8676-5a5e0eb071d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:28.282679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdaa28-a376-4c54-8676-5a5e0eb071d5 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:28.283155 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdaa28-a376-4c54-8676-5a5e0eb071d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:28.283596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdaa28-a376-4c54-8676-5a5e0eb071d5 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:28.284419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fdaa28-a376-4c54-8676-5a5e0eb071d5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:28.288623 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fdaa28-a376-4c54-8676-5a5e0eb071d5 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:28.301913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be42f2-22e3-4d37-afa9-33ca63f46296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:28.302683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be42f2-22e3-4d37-afa9-33ca63f46296 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:28.302968 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be42f2-22e3-4d37-afa9-33ca63f46296 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:28.303250 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be42f2-22e3-4d37-afa9-33ca63f46296 None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:28.304108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be42f2-22e3-4d37-afa9-33ca63f46296 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:28.304420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be42f2-22e3-4d37-afa9-33ca63f46296 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:28.305859 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be42f2-22e3-4d37-afa9-33ca63f46296 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:28.351083 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 368/735] 10.222.0.65 () {60 vars in 1141 bytes} [Tue May 10 19:58:28 2022] POST /identity/v3/services => generated 240 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:58:28.352285 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdaa28-a376-4c54-8676-5a5e0eb071d5 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:28.398168 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 368/736] 10.222.0.65 () {56 vars in 1133 bytes} [Tue May 10 19:58:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:28.403314 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be42f2-22e3-4d37-afa9-33ca63f46296 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:28.509315 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdaa28-a376-4c54-8676-5a5e0eb071d5 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:28.549380 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457dbea-cd20-4759-b172-cd9c08d07180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:28.550641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457dbea-cd20-4759-b172-cd9c08d07180 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:28.551162 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457dbea-cd20-4759-b172-cd9c08d07180 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:28.551719 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457dbea-cd20-4759-b172-cd9c08d07180 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:28.552764 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457dbea-cd20-4759-b172-cd9c08d07180 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:28.553930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457dbea-cd20-4759-b172-cd9c08d07180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:28.555828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457dbea-cd20-4759-b172-cd9c08d07180 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:28.567370 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 369/737] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:58:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:28.573041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457dbea-cd20-4759-b172-cd9c08d07180 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:28.634118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be42f2-22e3-4d37-afa9-33ca63f46296 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:28.678768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef94bc1-7d4b-4aca-addc-e27413d3ae98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:28.679933 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef94bc1-7d4b-4aca-addc-e27413d3ae98 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:28.680596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef94bc1-7d4b-4aca-addc-e27413d3ae98 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:28.681085 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef94bc1-7d4b-4aca-addc-e27413d3ae98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:28.694532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef94bc1-7d4b-4aca-addc-e27413d3ae98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:28.696901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef94bc1-7d4b-4aca-addc-e27413d3ae98 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:28.719378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-cef94bc1-7d4b-4aca-addc-e27413d3ae98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:28.719378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:28.719378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:28.719378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:28.719378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:28.719378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:28.719378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:28.719378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:28.719378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:28.719378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:28.719378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:28.719378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:28.719378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:28.719378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:28.719378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:28.720915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:28.720915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:28.720915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:28.720915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:28.720915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:28.720915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:28.720915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:28.720915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:28.720915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:28.720915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:28.720915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:28.720915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:28.720915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:28.720915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:28.720915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:28.720915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:28.720915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:28.721850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:28.721850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:28.721850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:28.721850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:28.722100 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457dbea-cd20-4759-b172-cd9c08d07180 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:28.725878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 369/738] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:28.731796 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef94bc1-7d4b-4aca-addc-e27413d3ae98 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:28.797586 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99358e54-8151-40dd-ba73-0c40aa2aa237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:28.798481 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99358e54-8151-40dd-ba73-0c40aa2aa237 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:28.800105 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99358e54-8151-40dd-ba73-0c40aa2aa237 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:28.800105 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99358e54-8151-40dd-ba73-0c40aa2aa237 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:28.820923 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99358e54-8151-40dd-ba73-0c40aa2aa237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:28.822417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99358e54-8151-40dd-ba73-0c40aa2aa237 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:28.856491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-99358e54-8151-40dd-ba73-0c40aa2aa237 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:28.856491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:28.856491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:28.856491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:28.856491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:28.856491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:28.856491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:28.856491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:28.856491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:28.856491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:28.856491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:28.856491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:28.856491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:28.856491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:28.856491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:28.858718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:28.858718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:28.858718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:28.858718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:28.858718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:28.858718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:28.858718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:28.858718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:28.858718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:28.858718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:28.858718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:28.858718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:28.858718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:28.858718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:28.858718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:28.858718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:28.858718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:28.861533 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:28.861533 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:28.861533 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:28.861533 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:28.861533 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 370/739] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:28.865286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99358e54-8151-40dd-ba73-0c40aa2aa237 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:28.893099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef94bc1-7d4b-4aca-addc-e27413d3ae98 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:28.932904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af93220-a1b1-4c65-b5d4-c8d2b74ac798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:28.934251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af93220-a1b1-4c65-b5d4-c8d2b74ac798 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:28.934535 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af93220-a1b1-4c65-b5d4-c8d2b74ac798 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:28.934809 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af93220-a1b1-4c65-b5d4-c8d2b74ac798 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:28.935553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af93220-a1b1-4c65-b5d4-c8d2b74ac798 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:28.935858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af93220-a1b1-4c65-b5d4-c8d2b74ac798 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:28.937226 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af93220-a1b1-4c65-b5d4-c8d2b74ac798 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:28.961878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 370/740] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:28.967790 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af93220-a1b1-4c65-b5d4-c8d2b74ac798 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:29.081847 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af93220-a1b1-4c65-b5d4-c8d2b74ac798 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:29.093933 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99358e54-8151-40dd-ba73-0c40aa2aa237 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:29.124650 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4c739-aa32-4406-a488-47c50a28663e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:29.125449 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4c739-aa32-4406-a488-47c50a28663e None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:29.125732 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4c739-aa32-4406-a488-47c50a28663e None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:29.126005 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4c739-aa32-4406-a488-47c50a28663e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:29.139045 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dbc1a-737b-4817-b1f2-8a4346a5dca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:29.139762 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dbc1a-737b-4817-b1f2-8a4346a5dca6 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:29.140049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dbc1a-737b-4817-b1f2-8a4346a5dca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:29.140322 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dbc1a-737b-4817-b1f2-8a4346a5dca6 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:29.141049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dbc1a-737b-4817-b1f2-8a4346a5dca6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:29.141353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dbc1a-737b-4817-b1f2-8a4346a5dca6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:29.143430 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dbc1a-737b-4817-b1f2-8a4346a5dca6 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:29.143718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4c739-aa32-4406-a488-47c50a28663e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:29.145160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4c739-aa32-4406-a488-47c50a28663e None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:29.156411 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 371/741] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:29.176564 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dbc1a-737b-4817-b1f2-8a4346a5dca6 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:29.180987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-46f4c739-aa32-4406-a488-47c50a28663e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:29.180987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:29.180987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:29.180987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:29.180987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:29.180987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:29.180987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:29.180987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:29.180987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:29.180987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:29.180987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:29.180987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:29.180987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:29.180987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:29.180987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:29.181870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:29.181870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:29.181870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:29.181870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:29.181870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:29.181870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:29.181870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:29.181870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:29.181870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:29.181870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:29.181870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:29.181870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:29.181870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:29.181870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:29.181870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:29.181870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:29.181870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:29.182810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:29.182810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:29.182810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:29.182810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:29.182810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:29.182810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:29.182810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:29.182810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:29.182810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 371/742] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:29.189218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4c739-aa32-4406-a488-47c50a28663e None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:29.289966 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dbc1a-737b-4817-b1f2-8a4346a5dca6 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:29.373851 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4c739-aa32-4406-a488-47c50a28663e None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:29.376425 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8614a19-1750-4394-a979-ceaadee27a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:29.378184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8614a19-1750-4394-a979-ceaadee27a93 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:29.378479 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8614a19-1750-4394-a979-ceaadee27a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:29.378761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8614a19-1750-4394-a979-ceaadee27a93 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:29.387027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8614a19-1750-4394-a979-ceaadee27a93 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:29.388827 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8614a19-1750-4394-a979-ceaadee27a93 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:29.408473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-f8614a19-1750-4394-a979-ceaadee27a93 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 10 19:58:29.408473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:29.408473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:29.408473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:29.408473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:29.408473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:29.408473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:29.408473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:29.408473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:29.408473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:29.408473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:29.408473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:29.408473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:29.408473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:29.408473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:29.408473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:29.409439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:29.409439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:29.409439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:29.409439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:29.409439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:29.409439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:29.409439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:29.409439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:29.409439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:29.409439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:29.409439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:29.409439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:29.409439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:29.409439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:29.409439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:29.409439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:29.409439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:29.409439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:29.410488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:29.410488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:29.410488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:29.410488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:29.410488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:29.410488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:29.410488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:29.410488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:29.410488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 10 19:58:29.410488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:29.414156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 372/743] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:58:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:29.415903 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214aa80-8999-4d69-8368-714a6a7e9698 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:29.418365 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214aa80-8999-4d69-8368-714a6a7e9698 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:29.418881 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214aa80-8999-4d69-8368-714a6a7e9698 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:29.422738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623a661-6bb2-4a87-8895-62e5fefbacd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:29.423200 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 373/744] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:29.424752 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214aa80-8999-4d69-8368-714a6a7e9698 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:29.425653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623a661-6bb2-4a87-8895-62e5fefbacd2 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:29.426598 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623a661-6bb2-4a87-8895-62e5fefbacd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:29.427491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623a661-6bb2-4a87-8895-62e5fefbacd2 None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:29.428516 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623a661-6bb2-4a87-8895-62e5fefbacd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:29.429056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623a661-6bb2-4a87-8895-62e5fefbacd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:29.431007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623a661-6bb2-4a87-8895-62e5fefbacd2 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:29.449984 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 372/745] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:29.452224 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54576d0f-df7d-4529-b988-46266e5a7dcc None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:29.452735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54576d0f-df7d-4529-b988-46266e5a7dcc None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:29.453283 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54576d0f-df7d-4529-b988-46266e5a7dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:29.475775 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-54576d0f-df7d-4529-b988-46266e5a7dcc None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:29.477364 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-54576d0f-df7d-4529-b988-46266e5a7dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:29.552245 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214aa80-8999-4d69-8368-714a6a7e9698 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:29.560983 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 373/746] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:29.562127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54576d0f-df7d-4529-b988-46266e5a7dcc None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:29.602666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fab7c7-8dcc-489f-93d2-1aabf0ccbac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:29.603772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fab7c7-8dcc-489f-93d2-1aabf0ccbac5 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:29.604302 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fab7c7-8dcc-489f-93d2-1aabf0ccbac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:29.604789 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fab7c7-8dcc-489f-93d2-1aabf0ccbac5 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:29.605890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fab7c7-8dcc-489f-93d2-1aabf0ccbac5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:29.606448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fab7c7-8dcc-489f-93d2-1aabf0ccbac5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:29.608152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fab7c7-8dcc-489f-93d2-1aabf0ccbac5 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:29.627792 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 374/747] 10.222.0.65 () {56 vars in 1135 bytes} [Tue May 10 19:58:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:29.628916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fab7c7-8dcc-489f-93d2-1aabf0ccbac5 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:29.679985 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54576d0f-df7d-4529-b988-46266e5a7dcc None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:29.715242 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c39ea-1519-464f-b264-9fb254b7d2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:29.716252 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c39ea-1519-464f-b264-9fb254b7d2b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:29.716468 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c39ea-1519-464f-b264-9fb254b7d2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:29.716791 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c39ea-1519-464f-b264-9fb254b7d2b1 None admin] PATH_INFO: `/v3/projects/a006477d1df94390b599621446bd0f02/users/ae922b2da4814d10bed1fa345700ac04/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:29.718843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c39ea-1519-464f-b264-9fb254b7d2b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=a006477d1df94390b599621446bd0f02, user_id=ae922b2da4814d10bed1fa345700ac04, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:29.720429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c39ea-1519-464f-b264-9fb254b7d2b1 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:29.732771 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fab7c7-8dcc-489f-93d2-1aabf0ccbac5 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:29.769795 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49374432-d8e8-4784-9eb2-03600c0fba09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:29.770620 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49374432-d8e8-4784-9eb2-03600c0fba09 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:29.770902 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49374432-d8e8-4784-9eb2-03600c0fba09 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:29.771174 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49374432-d8e8-4784-9eb2-03600c0fba09 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:29.781162 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49374432-d8e8-4784-9eb2-03600c0fba09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:29.788893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-49374432-d8e8-4784-9eb2-03600c0fba09 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:29.788893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:29.788893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:29.788893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:29.788893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:29.788893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:29.788893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:29.788893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:29.788893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:29.788893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:29.788893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:29.788893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:29.788893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:29.788893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:29.788893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:29.788893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:29.789842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:29.789842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:29.789842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:29.789842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:29.789842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:29.789842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:29.789842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:29.789842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:29.789842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:29.789842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:29.789842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:29.789842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:29.789842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:29.789842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:29.789842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:29.789842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:29.791588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:29.791588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:29.791588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:29.791588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:29.791588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:29.791588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:29.791588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:29.791588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:29.791588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:29.791588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:29.791588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:29.791588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:29.793231 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 375/748] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:29.794298 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49374432-d8e8-4784-9eb2-03600c0fba09 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:29.858363 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 374/749] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:58:29 2022] PUT /identity/v3/projects/a006477d1df94390b599621446bd0f02/users/ae922b2da4814d10bed1fa345700ac04/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:58:29.859803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c39ea-1519-464f-b264-9fb254b7d2b1 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:29.918895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49374432-d8e8-4784-9eb2-03600c0fba09 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:29.961743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d514e-5884-436d-9269-d3174f047738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:29.962593 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d514e-5884-436d-9269-d3174f047738 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:29.962871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d514e-5884-436d-9269-d3174f047738 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:29.963146 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d514e-5884-436d-9269-d3174f047738 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:30.043183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d514e-5884-436d-9269-d3174f047738 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:30.044807 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d514e-5884-436d-9269-d3174f047738 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:30.084335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c39ea-1519-464f-b264-9fb254b7d2b1 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:30.163165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e096773-4959-4d0f-8e41-3833f156fd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:30.163847 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e096773-4959-4d0f-8e41-3833f156fd76 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:30.164133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e096773-4959-4d0f-8e41-3833f156fd76 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:30.164412 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e096773-4959-4d0f-8e41-3833f156fd76 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:30.165846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e096773-4959-4d0f-8e41-3833f156fd76 None admin] RBAC: Adding query filter params (name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:30.166149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e096773-4959-4d0f-8e41-3833f156fd76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:30.168040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e096773-4959-4d0f-8e41-3833f156fd76 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:30.176106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 376/750] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:58:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 19:58:30.178013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f436801-5a3e-4463-aaf5-0e33b9314c66 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:30.179887 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f436801-5a3e-4463-aaf5-0e33b9314c66 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:30.179887 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f436801-5a3e-4463-aaf5-0e33b9314c66 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:30.181003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 377/751] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:30.181621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 375/752] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 19:58:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:30.181783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f436801-5a3e-4463-aaf5-0e33b9314c66 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:30.186787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c220cc-d540-449c-b7c4-cefa71f55163 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:30.187081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c220cc-d540-449c-b7c4-cefa71f55163 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:30.188139 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c220cc-d540-449c-b7c4-cefa71f55163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:30.204839 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-f4c220cc-d540-449c-b7c4-cefa71f55163 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:30.206013 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-f4c220cc-d540-449c-b7c4-cefa71f55163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:30.294727 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 376/753] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:58:30.295130 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f436801-5a3e-4463-aaf5-0e33b9314c66 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:30.301092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c220cc-d540-449c-b7c4-cefa71f55163 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:30.365373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc728c-4615-4a7d-a2a8-c96cbd045827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:30.366461 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc728c-4615-4a7d-a2a8-c96cbd045827 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:30.366741 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc728c-4615-4a7d-a2a8-c96cbd045827 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:30.367031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc728c-4615-4a7d-a2a8-c96cbd045827 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:30.380613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cc728c-4615-4a7d-a2a8-c96cbd045827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:30.382247 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cc728c-4615-4a7d-a2a8-c96cbd045827 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:30.405672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-79cc728c-4615-4a7d-a2a8-c96cbd045827 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:30.405672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:30.405672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:30.405672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:30.405672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:30.405672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:30.405672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:30.405672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:30.405672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:30.405672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:30.405672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:30.405672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:30.405672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:30.405672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:30.405672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:30.406708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:30.406708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:30.406708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:30.406708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:30.406708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:30.406708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:30.406708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:30.406708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:30.406708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:30.406708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:30.406708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:30.406708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:30.406708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:30.406708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:30.406708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:30.406708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:30.406708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:30.408156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:30.408156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:30.408156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:30.408156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:30.408156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 378/754] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:30.413135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc728c-4615-4a7d-a2a8-c96cbd045827 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:30.450126 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c220cc-d540-449c-b7c4-cefa71f55163 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:30.491002 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3983077-2a6b-4262-8a59-19f1c8432b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:30.492658 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3983077-2a6b-4262-8a59-19f1c8432b8f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:30.493131 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3983077-2a6b-4262-8a59-19f1c8432b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:30.493611 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3983077-2a6b-4262-8a59-19f1c8432b8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:30.507002 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3983077-2a6b-4262-8a59-19f1c8432b8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:30.509156 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3983077-2a6b-4262-8a59-19f1c8432b8f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:30.535135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc728c-4615-4a7d-a2a8-c96cbd045827 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:30.537415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-b3983077-2a6b-4262-8a59-19f1c8432b8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:30.537415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:30.537415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:30.537415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:30.537415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:30.537415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:30.537415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:30.537415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:30.537415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:30.537415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:30.537415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:30.537415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:30.537415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:30.537415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:30.537415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:30.538335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:30.538335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:30.538335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:30.538335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:30.538335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:30.538335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:30.538335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:30.538335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:30.538335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:30.538335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:30.538335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:30.538335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:30.538335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:30.538335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:30.538335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:30.538335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:30.538335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:30.539308 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:30.539308 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:30.539308 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:30.539308 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:30.540777 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 377/755] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:30.540777 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3983077-2a6b-4262-8a59-19f1c8432b8f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:30.578724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df93866-96b3-4304-8b86-29503ecf153a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:30.580031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df93866-96b3-4304-8b86-29503ecf153a None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:30.580318 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df93866-96b3-4304-8b86-29503ecf153a None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:30.580593 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df93866-96b3-4304-8b86-29503ecf153a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:30.581276 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df93866-96b3-4304-8b86-29503ecf153a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:30.582654 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df93866-96b3-4304-8b86-29503ecf153a None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:30.594909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-9df93866-96b3-4304-8b86-29503ecf153a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 19:58:30.594909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:30.594909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:30.594909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:30.594909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:30.594909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:30.594909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:30.594909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:30.594909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 19:58:30.594909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:30.594909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:30.594909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:30.594909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:30.594909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:30.594909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:30.594909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:30.596529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:30.596529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:30.596529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:30.596529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:30.596529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:30.596529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:30.596529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:30.596529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:30.596529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:30.596529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:30.596529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:30.596529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:30.596529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:30.596529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:30.596529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:30.596529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:30.597356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:30.597356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:30.597356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:30.597356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:30.597356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:30.597356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:30.597356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:30.597356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:30.597356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:30.597356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:30.597356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:30.597356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:30.597356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:30.597356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 19:58:30.597356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:30.601669 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 379/756] 10.222.0.65 () {56 vars in 1129 bytes} [Tue May 10 19:58:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:30.602612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df93866-96b3-4304-8b86-29503ecf153a None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:30.674999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3983077-2a6b-4262-8a59-19f1c8432b8f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:30.716490 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1d1df-429d-4560-b054-0cd5900dfd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:30.717571 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1d1df-429d-4560-b054-0cd5900dfd74 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:30.718065 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1d1df-429d-4560-b054-0cd5900dfd74 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:30.718630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1d1df-429d-4560-b054-0cd5900dfd74 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:30.719710 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1d1df-429d-4560-b054-0cd5900dfd74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:30.720870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1d1df-429d-4560-b054-0cd5900dfd74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:30.722628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1d1df-429d-4560-b054-0cd5900dfd74 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:30.741788 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 378/757] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:30.743243 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1d1df-429d-4560-b054-0cd5900dfd74 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:30.840845 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df93866-96b3-4304-8b86-29503ecf153a None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:30.855919 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1d1df-429d-4560-b054-0cd5900dfd74 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:30.885666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133469d0-8c0a-4cb0-8dac-2948db8f84d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:30.886981 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133469d0-8c0a-4cb0-8dac-2948db8f84d9 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:30.887942 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133469d0-8c0a-4cb0-8dac-2948db8f84d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:30.888704 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133469d0-8c0a-4cb0-8dac-2948db8f84d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:30.889946 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133469d0-8c0a-4cb0-8dac-2948db8f84d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:30.890479 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133469d0-8c0a-4cb0-8dac-2948db8f84d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:30.892790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133469d0-8c0a-4cb0-8dac-2948db8f84d9 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:30.893516 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8dbd9-6d2d-4fb6-8d87-1e478bbb8924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:30.894221 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8dbd9-6d2d-4fb6-8d87-1e478bbb8924 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:30.894506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8dbd9-6d2d-4fb6-8d87-1e478bbb8924 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:30.894784 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8dbd9-6d2d-4fb6-8d87-1e478bbb8924 None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:30.895507 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8dbd9-6d2d-4fb6-8d87-1e478bbb8924 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:30.895813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8dbd9-6d2d-4fb6-8d87-1e478bbb8924 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:30.897161 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8dbd9-6d2d-4fb6-8d87-1e478bbb8924 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:30.909207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 379/758] 10.222.0.65 () {56 vars in 1124 bytes} [Tue May 10 19:58:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:30.909207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8dbd9-6d2d-4fb6-8d87-1e478bbb8924 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:30.911689 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 380/759] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:30.913509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc922e8-b92c-4e7b-98c1-11cbfcb05cbe None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:30.916218 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc922e8-b92c-4e7b-98c1-11cbfcb05cbe None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:30.916218 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc922e8-b92c-4e7b-98c1-11cbfcb05cbe None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:30.916218 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 381/760] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:30.916849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc922e8-b92c-4e7b-98c1-11cbfcb05cbe None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:31.014811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8dbd9-6d2d-4fb6-8d87-1e478bbb8924 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:31.015006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc922e8-b92c-4e7b-98c1-11cbfcb05cbe None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:31.047954 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d7ef6-c41d-467d-8c30-3f6c65a5e078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:31.048622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d7ef6-c41d-467d-8c30-3f6c65a5e078 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:31.048900 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d7ef6-c41d-467d-8c30-3f6c65a5e078 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:31.049167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d7ef6-c41d-467d-8c30-3f6c65a5e078 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:31.049868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d7ef6-c41d-467d-8c30-3f6c65a5e078 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:31.050194 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d7ef6-c41d-467d-8c30-3f6c65a5e078 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:31.051541 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d7ef6-c41d-467d-8c30-3f6c65a5e078 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:31.052566 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a74ef-63e7-46fe-a8f0-e3f3a54611a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:31.053487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a74ef-63e7-46fe-a8f0-e3f3a54611a1 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:31.053771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a74ef-63e7-46fe-a8f0-e3f3a54611a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:31.054044 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a74ef-63e7-46fe-a8f0-e3f3a54611a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:31.057187 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 382/761] 10.222.0.65 () {56 vars in 1124 bytes} [Tue May 10 19:58:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:31.061305 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a74ef-63e7-46fe-a8f0-e3f3a54611a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:31.062423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d7ef6-c41d-467d-8c30-3f6c65a5e078 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:31.062851 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a74ef-63e7-46fe-a8f0-e3f3a54611a1 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:31.078366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-c91a74ef-63e7-46fe-a8f0-e3f3a54611a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:31.078366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:31.078366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:31.078366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:31.078366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:31.078366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:31.078366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:31.078366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:31.078366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:31.078366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:31.078366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:31.078366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:31.078366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:31.078366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:31.078366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:31.079275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:31.079275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:31.079275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:31.079275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:31.079275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:31.079275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:31.079275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:31.079275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:31.079275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:31.079275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:31.079275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:31.079275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:31.079275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:31.079275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:31.079275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:31.079275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:31.079275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:31.080825 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:31.080825 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:31.080825 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:31.080825 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:31.080825 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:31.080825 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:31.080825 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:31.080825 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:31.080825 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 380/762] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:31.082137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78dd10c-2120-4fe8-b3ad-851d012878ce None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:31.082137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78dd10c-2120-4fe8-b3ad-851d012878ce None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:31.082137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78dd10c-2120-4fe8-b3ad-851d012878ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:31.097914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-e78dd10c-2120-4fe8-b3ad-851d012878ce None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:31.099115 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-e78dd10c-2120-4fe8-b3ad-851d012878ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:31.167513 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 381/763] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:31.168733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78dd10c-2120-4fe8-b3ad-851d012878ce None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:31.184024 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d7ef6-c41d-467d-8c30-3f6c65a5e078 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:31.221792 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f95b75-a12c-4134-8164-36f5ac7a6a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:31.222870 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f95b75-a12c-4134-8164-36f5ac7a6a45 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:31.223150 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f95b75-a12c-4134-8164-36f5ac7a6a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:31.223420 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f95b75-a12c-4134-8164-36f5ac7a6a45 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:31.230911 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f95b75-a12c-4134-8164-36f5ac7a6a45 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:31.232418 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f95b75-a12c-4134-8164-36f5ac7a6a45 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:31.250861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-74f95b75-a12c-4134-8164-36f5ac7a6a45 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 19:58:31.250861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:31.250861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:31.250861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:31.250861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:31.250861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:31.250861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:31.250861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:31.250861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:31.250861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:31.250861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:31.250861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:31.250861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:31.250861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:31.250861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:31.250861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:31.251872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:31.251872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:31.251872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:31.251872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:31.251872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:31.251872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:31.251872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:31.251872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:31.251872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:31.251872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:31.251872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:31.251872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:31.251872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:31.251872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:31.251872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:31.251872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:31.251872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:31.251872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:31.252816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:31.252816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:31.252816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:31.252816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:31.252816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:31.252816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:31.252816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:31.252816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:31.252816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 19:58:31.252816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:31.256640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 383/764] 10.222.0.65 () {56 vars in 1117 bytes} [Tue May 10 19:58:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:31.264450 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f95b75-a12c-4134-8164-36f5ac7a6a45 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:31.279450 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78dd10c-2120-4fe8-b3ad-851d012878ce None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:31.314925 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5daf4e-92e5-4531-9806-974761ca0be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:31.315594 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5daf4e-92e5-4531-9806-974761ca0be1 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:31.315881 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5daf4e-92e5-4531-9806-974761ca0be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:31.316163 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5daf4e-92e5-4531-9806-974761ca0be1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:31.316967 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5daf4e-92e5-4531-9806-974761ca0be1 None admin] RBAC: Adding query filter params (interface=public, service_id=5d5651f2df1b418da867e3f2e070d985, region_id=RegionOne) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:31.317282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5daf4e-92e5-4531-9806-974761ca0be1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:31.318694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5daf4e-92e5-4531-9806-974761ca0be1 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:31.326066 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 382/765] 10.222.0.65 () {56 vars in 1251 bytes} [Tue May 10 19:58:31 2022] GET /identity/v3/endpoints?service_id=5d5651f2df1b418da867e3f2e070d985&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:31.326908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5daf4e-92e5-4531-9806-974761ca0be1 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:31.368857 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f95b75-a12c-4134-8164-36f5ac7a6a45 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:31.406967 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e330dae-2f06-4a2c-a1c1-53d84f1206e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:31.408170 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e330dae-2f06-4a2c-a1c1-53d84f1206e7 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:31.408457 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e330dae-2f06-4a2c-a1c1-53d84f1206e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:31.408743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e330dae-2f06-4a2c-a1c1-53d84f1206e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:31.409628 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e330dae-2f06-4a2c-a1c1-53d84f1206e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:31.409937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e330dae-2f06-4a2c-a1c1-53d84f1206e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:31.411384 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e330dae-2f06-4a2c-a1c1-53d84f1206e7 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:31.424668 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 384/766] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:31.428205 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e330dae-2f06-4a2c-a1c1-53d84f1206e7 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:31.434369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5daf4e-92e5-4531-9806-974761ca0be1 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:31.467343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460c868-706a-400b-bb0a-0a6260968672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:31.468134 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460c868-706a-400b-bb0a-0a6260968672 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:31.468419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460c868-706a-400b-bb0a-0a6260968672 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:31.468694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460c868-706a-400b-bb0a-0a6260968672 None admin] PATH_INFO: `/v3/services/network` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:31.469350 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460c868-706a-400b-bb0a-0a6260968672 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:31.471294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460c868-706a-400b-bb0a-0a6260968672 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:31.482913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-3460c868-706a-400b-bb0a-0a6260968672 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 10 19:58:31.482913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:31.482913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:31.482913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:31.482913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:31.482913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:31.482913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:31.482913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:31.482913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 10 19:58:31.482913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:31.482913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:31.482913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:31.482913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:31.482913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:31.482913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:31.482913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:31.482913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:31.483872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:31.483872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:31.483872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:31.483872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:31.483872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:31.483872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:31.483872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:31.483872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:31.483872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:31.483872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:31.483872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:31.483872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:31.483872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:31.483872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:31.483872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:31.483872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:31.484718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:31.484718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:31.484718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:31.484718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:31.484718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:31.484718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:31.484718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:31.484718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:31.484718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:31.484718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:31.484718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:31.484718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:31.484718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 10 19:58:31.484718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:31.484718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 383/767] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:31 2022] GET /identity/v3/services/network => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:31.485498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460c868-706a-400b-bb0a-0a6260968672 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:31.531593 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e330dae-2f06-4a2c-a1c1-53d84f1206e7 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:31.564564 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ba461-f30f-4415-84a3-5b488c928452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:31.565452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ba461-f30f-4415-84a3-5b488c928452 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:31.565877 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ba461-f30f-4415-84a3-5b488c928452 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:31.566312 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ba461-f30f-4415-84a3-5b488c928452 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:31.567331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ba461-f30f-4415-84a3-5b488c928452 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:31.567773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ba461-f30f-4415-84a3-5b488c928452 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:31.569322 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ba461-f30f-4415-84a3-5b488c928452 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:31.585497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460c868-706a-400b-bb0a-0a6260968672 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:31.587748 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 385/768] 10.222.0.65 () {56 vars in 1149 bytes} [Tue May 10 19:58:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:31.588722 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ba461-f30f-4415-84a3-5b488c928452 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:31.622825 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8fff2-094d-4cd3-bbd1-52bd5c548a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:31.623488 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8fff2-094d-4cd3-bbd1-52bd5c548a0f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:31.623773 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8fff2-094d-4cd3-bbd1-52bd5c548a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:31.624051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8fff2-094d-4cd3-bbd1-52bd5c548a0f None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:31.624749 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8fff2-094d-4cd3-bbd1-52bd5c548a0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:31.626121 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8fff2-094d-4cd3-bbd1-52bd5c548a0f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:31.631614 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 384/769] 10.222.0.65 () {56 vars in 1087 bytes} [Tue May 10 19:58:31 2022] GET /identity/v3/services => generated 772 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:31.632384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8fff2-094d-4cd3-bbd1-52bd5c548a0f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:31.705722 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ba461-f30f-4415-84a3-5b488c928452 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:31.743582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9ca0e-43ad-455c-af2d-60d83698bc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:31.744250 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9ca0e-43ad-455c-af2d-60d83698bc4d None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:31.744539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9ca0e-43ad-455c-af2d-60d83698bc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:31.744821 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9ca0e-43ad-455c-af2d-60d83698bc4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:31.745666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9ca0e-43ad-455c-af2d-60d83698bc4d None admin] RBAC: Adding query filter params (role.id=5c87e804d0764309ba08ad6e06b23a87, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=b4d10b258a53484da22950e2b1db2e41) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:31.745978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9ca0e-43ad-455c-af2d-60d83698bc4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:31.747560 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9ca0e-43ad-455c-af2d-60d83698bc4d None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:31.758547 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 386/770] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:58:31 2022] GET /identity/v3/role_assignments?user.id=b4d10b258a53484da22950e2b1db2e41&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:31.759600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9ca0e-43ad-455c-af2d-60d83698bc4d None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:31.775151 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8fff2-094d-4cd3-bbd1-52bd5c548a0f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:31.824069 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f016f53-7b49-4cfd-9714-96172ac943ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:31.825059 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f016f53-7b49-4cfd-9714-96172ac943ff None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:31.827285 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f016f53-7b49-4cfd-9714-96172ac943ff None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:31.827762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f016f53-7b49-4cfd-9714-96172ac943ff None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:31.828697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f016f53-7b49-4cfd-9714-96172ac943ff None admin] RBAC: Adding query filter params (name=network) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:31.829002 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f016f53-7b49-4cfd-9714-96172ac943ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:31.831056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f016f53-7b49-4cfd-9714-96172ac943ff None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:31.836393 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 385/771] 10.222.0.65 () {56 vars in 1112 bytes} [Tue May 10 19:58:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:31.843446 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f016f53-7b49-4cfd-9714-96172ac943ff None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:31.915597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9ca0e-43ad-455c-af2d-60d83698bc4d None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:31.953543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5356e-d1f5-46b8-8bd4-1d9a9b60ce2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:31.954192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5356e-d1f5-46b8-8bd4-1d9a9b60ce2f None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:31.955042 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5356e-d1f5-46b8-8bd4-1d9a9b60ce2f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:31.955324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5356e-d1f5-46b8-8bd4-1d9a9b60ce2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:31.957230 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f016f53-7b49-4cfd-9714-96172ac943ff None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:31.995940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b991e-4cc4-4150-8c69-01a1445b8cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:31.996690 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b991e-4cc4-4150-8c69-01a1445b8cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:31.996888 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b991e-4cc4-4150-8c69-01a1445b8cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:31.997160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b991e-4cc4-4150-8c69-01a1445b8cf5 None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:31.997887 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b991e-4cc4-4150-8c69-01a1445b8cf5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:32.005875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b991e-4cc4-4150-8c69-01a1445b8cf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:32.005875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b991e-4cc4-4150-8c69-01a1445b8cf5 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:32.005875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 386/772] 10.222.0.65 () {56 vars in 1112 bytes} [Tue May 10 19:58:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:32.011317 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b991e-4cc4-4150-8c69-01a1445b8cf5 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:32.047670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5356e-d1f5-46b8-8bd4-1d9a9b60ce2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:32.049715 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5356e-d1f5-46b8-8bd4-1d9a9b60ce2f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:32.120846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b991e-4cc4-4150-8c69-01a1445b8cf5 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:32.151331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 387/773] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:58:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 19:58:32.156149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8c461-4141-4d6f-ae8b-36a7ccf8f1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:32.156638 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5356e-d1f5-46b8-8bd4-1d9a9b60ce2f None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:32.157253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8c461-4141-4d6f-ae8b-36a7ccf8f1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:32.157754 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8c461-4141-4d6f-ae8b-36a7ccf8f1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:32.158261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8c461-4141-4d6f-ae8b-36a7ccf8f1a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:32.159249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d8c461-4141-4d6f-ae8b-36a7ccf8f1a8 None admin] RBAC: Adding query filter params (interface=public, service_id=fbaca6f4ba9648b88f3bbcb582b1084f, region_id=RegionOne) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:32.159765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d8c461-4141-4d6f-ae8b-36a7ccf8f1a8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:32.161313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d8c461-4141-4d6f-ae8b-36a7ccf8f1a8 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:32.167328 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 387/774] 10.222.0.65 () {56 vars in 1251 bytes} [Tue May 10 19:58:32 2022] GET /identity/v3/endpoints?service_id=fbaca6f4ba9648b88f3bbcb582b1084f&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:32.175156 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8c461-4141-4d6f-ae8b-36a7ccf8f1a8 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:32.320038 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5356e-d1f5-46b8-8bd4-1d9a9b60ce2f None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:32.358438 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784fd35-bee5-4e3f-8ba6-40e50bae5a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:32.359695 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784fd35-bee5-4e3f-8ba6-40e50bae5a2b None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:32.360300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784fd35-bee5-4e3f-8ba6-40e50bae5a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:32.361318 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784fd35-bee5-4e3f-8ba6-40e50bae5a2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:32.373418 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784fd35-bee5-4e3f-8ba6-40e50bae5a2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:32.375383 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784fd35-bee5-4e3f-8ba6-40e50bae5a2b None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:32.393773 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8c461-4141-4d6f-ae8b-36a7ccf8f1a8 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:32.397421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-b784fd35-bee5-4e3f-8ba6-40e50bae5a2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:32.397421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:32.397421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:32.397421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:32.397421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:32.397421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:32.397421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:32.397421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:32.397421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:32.397421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:32.397421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:32.397421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:32.397421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:32.397421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:32.397421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:32.398361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:32.398361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:32.398361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:32.398361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:32.398361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:32.398361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:32.398361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:32.398361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:32.398361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:32.398361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:32.398361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:32.398361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:32.398361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:32.398361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:32.398361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:32.398361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:32.398361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:32.399271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:32.399271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:32.399271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:32.399271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:32.402040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 388/775] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:32.403618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465b365-94c5-496d-ae6b-54fc253fd147 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:32.404075 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465b365-94c5-496d-ae6b-54fc253fd147 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:32.404518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465b365-94c5-496d-ae6b-54fc253fd147 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:32.406134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 389/776] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:32.407051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465b365-94c5-496d-ae6b-54fc253fd147 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:32.441073 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7ad68-0188-4804-8a64-2c08cfc1700f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:32.442112 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7ad68-0188-4804-8a64-2c08cfc1700f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:32.442647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7ad68-0188-4804-8a64-2c08cfc1700f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:32.443142 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7ad68-0188-4804-8a64-2c08cfc1700f None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:32.444089 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7ad68-0188-4804-8a64-2c08cfc1700f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:32.445663 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7ad68-0188-4804-8a64-2c08cfc1700f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:32.451902 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 388/777] 10.222.0.65 () {56 vars in 1087 bytes} [Tue May 10 19:58:32 2022] GET /identity/v3/services => generated 772 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:32.453399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c1407-94d3-4c5d-ad1e-849fb2735573 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:32.453938 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c1407-94d3-4c5d-ad1e-849fb2735573 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:32.454500 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c1407-94d3-4c5d-ad1e-849fb2735573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:32.472909 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-d87c1407-94d3-4c5d-ad1e-849fb2735573 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:32.474467 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-d87c1407-94d3-4c5d-ad1e-849fb2735573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:32.525155 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465b365-94c5-496d-ae6b-54fc253fd147 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:32.547487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 389/778] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:32.566268 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac81db9-4ba9-48a9-81f7-780acc679a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:32.566998 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac81db9-4ba9-48a9-81f7-780acc679a02 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:32.567284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac81db9-4ba9-48a9-81f7-780acc679a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:32.567564 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac81db9-4ba9-48a9-81f7-780acc679a02 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:32.568361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac81db9-4ba9-48a9-81f7-780acc679a02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:32.568728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac81db9-4ba9-48a9-81f7-780acc679a02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:32.570191 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac81db9-4ba9-48a9-81f7-780acc679a02 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:32.582901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c1407-94d3-4c5d-ad1e-849fb2735573 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:32.587464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 390/779] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:32.588669 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac81db9-4ba9-48a9-81f7-780acc679a02 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:32.725577 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c1407-94d3-4c5d-ad1e-849fb2735573 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:32.772866 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c5986-0065-45b4-9a47-429f4da72a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:32.774129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c5986-0065-45b4-9a47-429f4da72a51 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:32.775414 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c5986-0065-45b4-9a47-429f4da72a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:32.775783 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c5986-0065-45b4-9a47-429f4da72a51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:32.789585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c5986-0065-45b4-9a47-429f4da72a51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:32.790765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac81db9-4ba9-48a9-81f7-780acc679a02 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:32.798278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-015c5986-0065-45b4-9a47-429f4da72a51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:58:32.798278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:32.798278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:32.798278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:32.798278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:32.798278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:32.798278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:32.798278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:32.798278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:32.798278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:32.798278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:32.798278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:32.798278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:32.798278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:32.798278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:32.798278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:32.799229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:32.799229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:32.799229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:32.799229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:32.799229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:32.799229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:32.799229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:32.799229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:32.799229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:32.799229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:32.799229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:32.799229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:32.799229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:32.799229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:32.799229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:32.799229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:32.800068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:32.800068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:32.800068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:32.800068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:32.800068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:32.800068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:32.800068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:32.800068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:32.800068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:32.800068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:32.800068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:58:32.800068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:32.800068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 390/780] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:58:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:32.806734 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c5986-0065-45b4-9a47-429f4da72a51 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:32.831757 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e26ea-8441-4298-be0a-35fb9a08885b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:32.832905 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e26ea-8441-4298-be0a-35fb9a08885b None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:32.833402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e26ea-8441-4298-be0a-35fb9a08885b None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:32.833877 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e26ea-8441-4298-be0a-35fb9a08885b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:32.841473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e26ea-8441-4298-be0a-35fb9a08885b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:32.843186 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e26ea-8441-4298-be0a-35fb9a08885b None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:32.858421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-7b6e26ea-8441-4298-be0a-35fb9a08885b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:32.858421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:32.858421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:32.858421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:32.858421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:32.858421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:32.858421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:32.858421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:32.858421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:32.858421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:32.858421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:32.858421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:32.858421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:32.858421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:32.858421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:32.859304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:32.859304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:32.859304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:32.859304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:32.859304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:32.859304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:32.859304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:32.859304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:32.859304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:32.859304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:32.859304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:32.859304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:32.859304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:32.859304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:32.859304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:32.859304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:32.859304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:32.860168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:32.860168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:32.860168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:32.860168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:32.860168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:32.860168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:32.860168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:32.860168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:32.862641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 391/781] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:32.874112 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e26ea-8441-4298-be0a-35fb9a08885b None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:32.915742 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c5986-0065-45b4-9a47-429f4da72a51 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:32.958990 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56955f2a-a85d-497b-8980-29ebfbff78ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:32.963639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56955f2a-a85d-497b-8980-29ebfbff78ae None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:32.963924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56955f2a-a85d-497b-8980-29ebfbff78ae None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:32.964201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56955f2a-a85d-497b-8980-29ebfbff78ae None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:32.964965 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56955f2a-a85d-497b-8980-29ebfbff78ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:32.965271 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56955f2a-a85d-497b-8980-29ebfbff78ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:32.968521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56955f2a-a85d-497b-8980-29ebfbff78ae None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:32.981935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 391/782] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:58:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:32.988244 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56955f2a-a85d-497b-8980-29ebfbff78ae None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:33.004580 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e26ea-8441-4298-be0a-35fb9a08885b None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:33.044808 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec20b1-b452-4cee-af46-70c8812a6efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:33.045608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec20b1-b452-4cee-af46-70c8812a6efd None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:33.045897 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec20b1-b452-4cee-af46-70c8812a6efd None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:33.046204 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec20b1-b452-4cee-af46-70c8812a6efd None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:33.047011 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec20b1-b452-4cee-af46-70c8812a6efd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:33.047316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec20b1-b452-4cee-af46-70c8812a6efd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:33.048957 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec20b1-b452-4cee-af46-70c8812a6efd None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:33.062886 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 392/783] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:33.068847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec20b1-b452-4cee-af46-70c8812a6efd None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:33.186348 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec20b1-b452-4cee-af46-70c8812a6efd None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:33.214771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56955f2a-a85d-497b-8980-29ebfbff78ae None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:33.236099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774de8c-3797-43d9-b44f-d10aa8d32c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:33.237215 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774de8c-3797-43d9-b44f-d10aa8d32c5c None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:33.238544 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774de8c-3797-43d9-b44f-d10aa8d32c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:33.239098 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774de8c-3797-43d9-b44f-d10aa8d32c5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:33.239970 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774de8c-3797-43d9-b44f-d10aa8d32c5c None admin] RBAC: Adding query filter params (role.id=55528a93cfa64162b2f9574fb5cdde79, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=7ec2605fd1b84e0ea134b943ed33a605) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:33.240285 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774de8c-3797-43d9-b44f-d10aa8d32c5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:33.241727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774de8c-3797-43d9-b44f-d10aa8d32c5c None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:33.266111 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 393/784] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:58:33 2022] GET /identity/v3/role_assignments?user.id=7ec2605fd1b84e0ea134b943ed33a605&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=55528a93cfa64162b2f9574fb5cdde79 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:33.269045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bcf39-fb80-49ae-af2b-b7f4620d3bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:33.269918 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bcf39-fb80-49ae-af2b-b7f4620d3bae None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:33.270233 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bcf39-fb80-49ae-af2b-b7f4620d3bae None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:33.270508 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bcf39-fb80-49ae-af2b-b7f4620d3bae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:33.288611 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991bcf39-fb80-49ae-af2b-b7f4620d3bae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:33.290160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991bcf39-fb80-49ae-af2b-b7f4620d3bae None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:33.317926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-991bcf39-fb80-49ae-af2b-b7f4620d3bae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:33.317926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:33.317926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:33.317926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:33.317926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:33.317926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:33.317926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:33.317926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:33.317926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:33.317926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:33.317926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:33.317926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:33.317926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:33.317926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:33.317926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:33.318930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:33.318930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:33.318930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:33.318930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:33.318930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:33.318930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:33.318930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:33.318930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:33.318930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:33.318930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:33.318930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:33.318930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:33.318930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:33.318930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:33.318930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:33.318930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:33.318930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:33.319830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:33.319830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:33.319830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:33.319830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:33.319830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 392/785] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:33.326887 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774de8c-3797-43d9-b44f-d10aa8d32c5c None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:33.473859 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774de8c-3797-43d9-b44f-d10aa8d32c5c None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:33.511307 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dadbdd-05cc-45f2-913d-492d8e57803e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:33.511899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dadbdd-05cc-45f2-913d-492d8e57803e None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:33.512186 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dadbdd-05cc-45f2-913d-492d8e57803e None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:33.512468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dadbdd-05cc-45f2-913d-492d8e57803e None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:33.513202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dadbdd-05cc-45f2-913d-492d8e57803e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:33.513794 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dadbdd-05cc-45f2-913d-492d8e57803e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:33.515198 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dadbdd-05cc-45f2-913d-492d8e57803e None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:33.527988 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 394/786] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:33.533320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bcf39-fb80-49ae-af2b-b7f4620d3bae None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:33.645280 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bcf39-fb80-49ae-af2b-b7f4620d3bae None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:33.691770 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5d6ad-3387-4012-8a61-eceddcf3b0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:33.692575 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5d6ad-3387-4012-8a61-eceddcf3b0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:33.692859 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5d6ad-3387-4012-8a61-eceddcf3b0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:33.693134 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5d6ad-3387-4012-8a61-eceddcf3b0b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:33.707262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5d6ad-3387-4012-8a61-eceddcf3b0b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:33.708735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5d6ad-3387-4012-8a61-eceddcf3b0b4 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:33.726562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-4bf5d6ad-3387-4012-8a61-eceddcf3b0b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 19:58:33.726562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:33.726562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:33.726562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:33.726562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:33.726562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:33.726562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:33.726562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:33.726562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:33.726562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:33.726562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:33.726562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:33.726562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:33.726562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:33.726562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:33.726562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:33.727532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:33.727532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:33.727532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:33.727532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:33.727532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:33.727532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:33.727532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:33.727532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:33.727532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:33.727532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:33.727532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:33.727532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:33.727532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:33.727532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:33.727532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:33.727532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:33.727532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:33.727532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:33.728570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:33.728570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:33.728570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:33.728570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:33.728570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:33.728570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:33.728570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:33.728570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:33.728570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 19:58:33.728570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:33.728570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 393/787] 10.222.0.65 () {56 vars in 1093 bytes} [Tue May 10 19:58:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:33.733567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dadbdd-05cc-45f2-913d-492d8e57803e None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:33.874701 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dadbdd-05cc-45f2-913d-492d8e57803e None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:33.911089 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d01aa-b72f-41f2-a2b7-6d4f7d2b3b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:33.911770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d01aa-b72f-41f2-a2b7-6d4f7d2b3b59 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:33.912054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d01aa-b72f-41f2-a2b7-6d4f7d2b3b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:33.912332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d01aa-b72f-41f2-a2b7-6d4f7d2b3b59 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:33.913186 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d01aa-b72f-41f2-a2b7-6d4f7d2b3b59 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:33.913494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d01aa-b72f-41f2-a2b7-6d4f7d2b3b59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:33.914986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d01aa-b72f-41f2-a2b7-6d4f7d2b3b59 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:33.940816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 395/788] 10.222.0.65 () {56 vars in 1133 bytes} [Tue May 10 19:58:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:33.946524 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5d6ad-3387-4012-8a61-eceddcf3b0b4 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:34.064187 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5d6ad-3387-4012-8a61-eceddcf3b0b4 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:34.098445 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06136067-5889-4e76-ab2b-1ca668304ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:34.099101 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06136067-5889-4e76-ab2b-1ca668304ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:34.099391 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06136067-5889-4e76-ab2b-1ca668304ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:34.099666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06136067-5889-4e76-ab2b-1ca668304ba6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:34.171769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06136067-5889-4e76-ab2b-1ca668304ba6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:34.173825 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06136067-5889-4e76-ab2b-1ca668304ba6 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:34.276575 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 394/789] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:58:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 19:58:34.283825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d01aa-b72f-41f2-a2b7-6d4f7d2b3b59 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:34.405984 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d01aa-b72f-41f2-a2b7-6d4f7d2b3b59 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:34.444752 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c08b3-ab67-4b79-9024-15de99817e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:34.445549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c08b3-ab67-4b79-9024-15de99817e2f None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:34.445831 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c08b3-ab67-4b79-9024-15de99817e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:34.446118 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c08b3-ab67-4b79-9024-15de99817e2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:34.460437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c08b3-ab67-4b79-9024-15de99817e2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:34.461892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c08b3-ab67-4b79-9024-15de99817e2f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:34.488224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-876c08b3-ab67-4b79-9024-15de99817e2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:34.488224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:34.488224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:34.488224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:34.488224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:34.488224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:34.488224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:34.488224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:34.488224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:34.488224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:34.488224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:34.488224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:34.488224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:34.488224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:34.488224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:34.489386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:34.489386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:34.489386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:34.489386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:34.489386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:34.489386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:34.489386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:34.489386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:34.489386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:34.489386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:34.489386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:34.489386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:34.489386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:34.489386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:34.489386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:34.489386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:34.489386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:34.490297 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:34.490297 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:34.490297 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:34.490297 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:34.490297 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 396/790] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:34.495480 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06136067-5889-4e76-ab2b-1ca668304ba6 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:34.507294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bc1e5-6cc2-4b9b-b2e9-63d16af7e449 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:34.507564 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bc1e5-6cc2-4b9b-b2e9-63d16af7e449 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:34.507847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bc1e5-6cc2-4b9b-b2e9-63d16af7e449 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:34.513380 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 397/791] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:34.514547 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291568f-6164-4bcd-a1ac-3a93a36035b7 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:34.517393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291568f-6164-4bcd-a1ac-3a93a36035b7 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:34.517689 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291568f-6164-4bcd-a1ac-3a93a36035b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:34.536484 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-1291568f-6164-4bcd-a1ac-3a93a36035b7 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:34.537822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1291568f-6164-4bcd-a1ac-3a93a36035b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:34.617256 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 398/792] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:34.652077 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291568f-6164-4bcd-a1ac-3a93a36035b7 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:34.669462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06136067-5889-4e76-ab2b-1ca668304ba6 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:34.712236 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc5f06-a90e-4ff4-8e0b-a25b2dddbdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:34.713142 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc5f06-a90e-4ff4-8e0b-a25b2dddbdbc None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:34.713596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc5f06-a90e-4ff4-8e0b-a25b2dddbdbc None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:34.714020 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc5f06-a90e-4ff4-8e0b-a25b2dddbdbc None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:34.714919 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc5f06-a90e-4ff4-8e0b-a25b2dddbdbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:34.715372 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc5f06-a90e-4ff4-8e0b-a25b2dddbdbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:34.716868 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc5f06-a90e-4ff4-8e0b-a25b2dddbdbc None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:34.737654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 395/793] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:34.743089 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc5f06-a90e-4ff4-8e0b-a25b2dddbdbc None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:34.755449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291568f-6164-4bcd-a1ac-3a93a36035b7 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:34.792849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4f1a3-31ed-4ceb-ad79-8d9dcc6bfec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:34.793737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4f1a3-31ed-4ceb-ad79-8d9dcc6bfec1 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:34.794021 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4f1a3-31ed-4ceb-ad79-8d9dcc6bfec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:34.794328 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4f1a3-31ed-4ceb-ad79-8d9dcc6bfec1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:34.794999 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4f1a3-31ed-4ceb-ad79-8d9dcc6bfec1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:34.798101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4f1a3-31ed-4ceb-ad79-8d9dcc6bfec1 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:34.808402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-55d4f1a3-31ed-4ceb-ad79-8d9dcc6bfec1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 19:58:34.808402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:34.808402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:34.808402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:34.808402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:34.808402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:34.808402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:34.808402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:34.808402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 19:58:34.808402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:34.808402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:34.808402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:34.808402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:34.808402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:34.808402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:34.808402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:34.809330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:34.809330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:34.809330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:34.809330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:34.809330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:34.809330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:34.809330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:34.809330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:34.809330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:34.809330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:34.809330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:34.809330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:34.809330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:34.809330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:34.809330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:34.809330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:34.810195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:34.810195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:34.810195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:34.810195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:34.810195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:34.810195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:34.810195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:34.810195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:34.810195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:34.810195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:34.810195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:34.810195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:34.810195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:34.810195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 19:58:34.810195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:34.810195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 399/794] 10.222.0.65 () {56 vars in 1129 bytes} [Tue May 10 19:58:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:34.811525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a049e-0b6f-49dd-bed3-82b89bc976ee None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:34.811525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a049e-0b6f-49dd-bed3-82b89bc976ee None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:34.811694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a049e-0b6f-49dd-bed3-82b89bc976ee None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:34.813120 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 400/795] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:34.819228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a049e-0b6f-49dd-bed3-82b89bc976ee None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:34.891431 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc5f06-a90e-4ff4-8e0b-a25b2dddbdbc None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:34.929235 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b44f63-491c-4ff6-99a4-d0f0c675e82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:34.930028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b44f63-491c-4ff6-99a4-d0f0c675e82f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:34.930417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b44f63-491c-4ff6-99a4-d0f0c675e82f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:34.930754 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b44f63-491c-4ff6-99a4-d0f0c675e82f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:34.936051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a049e-0b6f-49dd-bed3-82b89bc976ee None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:34.938454 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b44f63-491c-4ff6-99a4-d0f0c675e82f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:34.940524 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b44f63-491c-4ff6-99a4-d0f0c675e82f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:34.955768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-d3b44f63-491c-4ff6-99a4-d0f0c675e82f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:34.955768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:34.955768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:34.955768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:34.955768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:34.955768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:34.955768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:34.955768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:34.955768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:34.955768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:34.955768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:34.955768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:34.955768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:34.955768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:34.955768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:34.957684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:34.957684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:34.957684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:34.957684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:34.957684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:34.957684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:34.957684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:34.957684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:34.957684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:34.957684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:34.957684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:34.957684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:34.957684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:34.957684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:34.957684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:34.957684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:34.957684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:34.959824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:34.959824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:34.959824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:34.959824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:34.959824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:34.959824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:34.959824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:34.959824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:34.959824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 396/796] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:34.959824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887d88c-79d8-4150-a207-ea93d9106f2f None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:34.959824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887d88c-79d8-4150-a207-ea93d9106f2f None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:34.961212 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887d88c-79d8-4150-a207-ea93d9106f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:34.976786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-3887d88c-79d8-4150-a207-ea93d9106f2f None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:34.977987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3887d88c-79d8-4150-a207-ea93d9106f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:34.978308 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba979d-35db-4e67-a08e-4eaee1f73037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:34.978981 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba979d-35db-4e67-a08e-4eaee1f73037 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:34.979323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba979d-35db-4e67-a08e-4eaee1f73037 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:34.979605 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba979d-35db-4e67-a08e-4eaee1f73037 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:34.981321 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba979d-35db-4e67-a08e-4eaee1f73037 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:34.981696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba979d-35db-4e67-a08e-4eaee1f73037 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:34.983249 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba979d-35db-4e67-a08e-4eaee1f73037 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:34.990355 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 401/797] 10.222.0.65 () {56 vars in 1124 bytes} [Tue May 10 19:58:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:34.991163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba979d-35db-4e67-a08e-4eaee1f73037 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:35.058429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 397/798] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:35.059513 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887d88c-79d8-4150-a207-ea93d9106f2f None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:35.103739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba979d-35db-4e67-a08e-4eaee1f73037 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:35.139208 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe3ed4-557b-47ab-84ec-d9be37003f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:35.139868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe3ed4-557b-47ab-84ec-d9be37003f8e None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:35.140155 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe3ed4-557b-47ab-84ec-d9be37003f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:35.140433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe3ed4-557b-47ab-84ec-d9be37003f8e None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:35.141228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe3ed4-557b-47ab-84ec-d9be37003f8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:35.141540 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe3ed4-557b-47ab-84ec-d9be37003f8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:35.143024 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe3ed4-557b-47ab-84ec-d9be37003f8e None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:35.155506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 402/799] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:35.156148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe3ed4-557b-47ab-84ec-d9be37003f8e None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:35.176318 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887d88c-79d8-4150-a207-ea93d9106f2f None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:35.215103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e024268-89fe-4ffb-b674-6beec92007f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:35.215772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e024268-89fe-4ffb-b674-6beec92007f8 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:35.216053 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e024268-89fe-4ffb-b674-6beec92007f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:35.216326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e024268-89fe-4ffb-b674-6beec92007f8 None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:35.217042 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e024268-89fe-4ffb-b674-6beec92007f8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:35.217348 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e024268-89fe-4ffb-b674-6beec92007f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:35.218759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e024268-89fe-4ffb-b674-6beec92007f8 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:35.228339 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 398/800] 10.222.0.65 () {56 vars in 1124 bytes} [Tue May 10 19:58:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:35.229347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e024268-89fe-4ffb-b674-6beec92007f8 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:35.271026 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe3ed4-557b-47ab-84ec-d9be37003f8e None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:35.309349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45019555-3543-4256-ac31-3fa15c784c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:35.310280 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45019555-3543-4256-ac31-3fa15c784c11 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:35.310558 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45019555-3543-4256-ac31-3fa15c784c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:35.310827 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45019555-3543-4256-ac31-3fa15c784c11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:35.321567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45019555-3543-4256-ac31-3fa15c784c11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:35.329836 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e024268-89fe-4ffb-b674-6beec92007f8 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:35.330317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-45019555-3543-4256-ac31-3fa15c784c11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:58:35.330317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:35.330317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:35.330317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:35.330317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:35.330317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:35.330317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:35.330317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:35.330317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:35.330317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:35.330317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:35.330317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:35.330317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:35.330317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:35.330317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:35.330317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:35.331201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:35.331201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:35.331201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:35.331201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:35.331201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:35.331201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:35.331201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:35.331201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:35.331201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:35.331201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:35.331201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:35.331201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:35.331201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:35.331201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:35.331201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:35.331201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:35.332574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:35.332574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:35.332574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:35.332574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:35.332574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:35.332574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:35.332574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:35.332574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:35.332574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:35.332574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:35.332574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:58:35.332574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:35.332574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 403/801] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:58:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:35.332574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45019555-3543-4256-ac31-3fa15c784c11 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:35.367369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7525c5-b604-4628-874e-7f207923a65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:35.368353 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7525c5-b604-4628-874e-7f207923a65d None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:35.368708 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7525c5-b604-4628-874e-7f207923a65d None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:35.369050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7525c5-b604-4628-874e-7f207923a65d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:35.369951 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7525c5-b604-4628-874e-7f207923a65d None admin] RBAC: Adding query filter params (role.id=5c87e804d0764309ba08ad6e06b23a87, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=ae922b2da4814d10bed1fa345700ac04) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:35.370370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7525c5-b604-4628-874e-7f207923a65d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:35.371857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7525c5-b604-4628-874e-7f207923a65d None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:35.378248 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 399/802] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:58:35 2022] GET /identity/v3/role_assignments?user.id=ae922b2da4814d10bed1fa345700ac04&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:35.379680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf5ef1-c152-4031-aeaa-9ac7e4b61f07 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:35.380036 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf5ef1-c152-4031-aeaa-9ac7e4b61f07 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:35.380400 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf5ef1-c152-4031-aeaa-9ac7e4b61f07 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:35.386054 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 400/803] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:35.387413 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf5ef1-c152-4031-aeaa-9ac7e4b61f07 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:35.464269 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45019555-3543-4256-ac31-3fa15c784c11 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:35.501899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7f99b-159c-45d3-8388-597938421311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:35.502296 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7f99b-159c-45d3-8388-597938421311 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:35.502585 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7f99b-159c-45d3-8388-597938421311 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:35.502865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7f99b-159c-45d3-8388-597938421311 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:35.503634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7f99b-159c-45d3-8388-597938421311 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:35.505336 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7f99b-159c-45d3-8388-597938421311 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:35.534534 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf5ef1-c152-4031-aeaa-9ac7e4b61f07 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:35.572535 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20587bbb-47f0-4752-a86d-5ace9f9d169c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:35.573721 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20587bbb-47f0-4752-a86d-5ace9f9d169c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:35.573721 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20587bbb-47f0-4752-a86d-5ace9f9d169c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:35.573911 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20587bbb-47f0-4752-a86d-5ace9f9d169c None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:35.575468 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20587bbb-47f0-4752-a86d-5ace9f9d169c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:35.575468 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20587bbb-47f0-4752-a86d-5ace9f9d169c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:35.576208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20587bbb-47f0-4752-a86d-5ace9f9d169c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:35.585537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 401/804] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:58:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:35.587053 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180efc7d-ccfe-4b07-a0e5-e0d09fcb396e None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:35.587163 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180efc7d-ccfe-4b07-a0e5-e0d09fcb396e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:35.587525 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180efc7d-ccfe-4b07-a0e5-e0d09fcb396e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:35.604533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-180efc7d-ccfe-4b07-a0e5-e0d09fcb396e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:35.605725 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-180efc7d-ccfe-4b07-a0e5-e0d09fcb396e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:35.609200 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 404/805] 10.222.0.65 () {60 vars in 1144 bytes} [Tue May 10 19:58:35 2022] POST /identity/v3/endpoints => generated 346 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:58:35.611252 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7f99b-159c-45d3-8388-597938421311 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:35.698119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 402/806] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:35.723180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180efc7d-ccfe-4b07-a0e5-e0d09fcb396e None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:35.751376 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7f99b-159c-45d3-8388-597938421311 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:35.794932 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281089f-213d-449c-94d6-c900cebb54a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:35.796104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281089f-213d-449c-94d6-c900cebb54a4 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:35.796592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281089f-213d-449c-94d6-c900cebb54a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:35.797072 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281089f-213d-449c-94d6-c900cebb54a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:35.810735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281089f-213d-449c-94d6-c900cebb54a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:35.812601 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281089f-213d-449c-94d6-c900cebb54a4 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:35.835549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-4281089f-213d-449c-94d6-c900cebb54a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:35.835549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:35.835549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:35.835549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:35.835549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:35.835549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:35.835549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:35.835549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:35.835549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:35.835549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:35.835549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:35.835549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:35.835549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:35.835549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:35.835549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:35.836494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:35.836494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:35.836494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:35.836494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:35.836494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:35.836494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:35.836494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:35.836494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:35.836494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:35.836494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:35.836494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:35.836494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:35.836494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:35.836494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:35.836494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:35.836494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:35.836494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:35.837376 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:35.837376 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:35.837376 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:35.837376 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:35.840090 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 405/807] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:35.847446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281089f-213d-449c-94d6-c900cebb54a4 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:35.880456 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180efc7d-ccfe-4b07-a0e5-e0d09fcb396e None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:35.919411 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b988d00-ad21-477a-9d92-38da7bc3a7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:35.920445 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b988d00-ad21-477a-9d92-38da7bc3a7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:35.920802 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b988d00-ad21-477a-9d92-38da7bc3a7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:35.921172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b988d00-ad21-477a-9d92-38da7bc3a7b8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:35.921933 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b988d00-ad21-477a-9d92-38da7bc3a7b8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:35.923528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b988d00-ad21-477a-9d92-38da7bc3a7b8 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:35.935765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-9b988d00-ad21-477a-9d92-38da7bc3a7b8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 10 19:58:35.935765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:35.935765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:35.935765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:35.935765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:35.935765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:35.935765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:35.935765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:35.935765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 10 19:58:35.935765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:35.935765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:35.935765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:35.935765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:35.935765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:35.935765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:35.935765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:35.935765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:35.936892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:35.936892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:35.936892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:35.936892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:35.936892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:35.936892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:35.936892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:35.936892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:35.936892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:35.936892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:35.936892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:35.936892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:35.936892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:35.936892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:35.936892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:35.936892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:35.937729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:35.937729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:35.937729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:35.937729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:35.937729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:35.937729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:35.937729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:35.937729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:35.937729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:35.937729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:35.937729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:35.937729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:35.937729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 10 19:58:35.937729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:35.937729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 403/808] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:35 2022] GET /identity/v3/services/network => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:35.945749 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b988d00-ad21-477a-9d92-38da7bc3a7b8 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:35.977791 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281089f-213d-449c-94d6-c900cebb54a4 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:36.024227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60c108-5523-4706-b8ff-04045846d081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:36.025244 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60c108-5523-4706-b8ff-04045846d081 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:36.025875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60c108-5523-4706-b8ff-04045846d081 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:36.026398 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60c108-5523-4706-b8ff-04045846d081 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:36.027823 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60c108-5523-4706-b8ff-04045846d081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:36.028529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60c108-5523-4706-b8ff-04045846d081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:36.031079 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60c108-5523-4706-b8ff-04045846d081 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:36.050238 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 406/809] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:36.052608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81d33c-abdb-42d9-b9a5-2befc2c9a1d2 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:36.052899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81d33c-abdb-42d9-b9a5-2befc2c9a1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:36.053178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81d33c-abdb-42d9-b9a5-2befc2c9a1d2 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:36.055788 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 407/810] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:36.055788 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81d33c-abdb-42d9-b9a5-2befc2c9a1d2 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:36.069963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b988d00-ad21-477a-9d92-38da7bc3a7b8 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:36.105780 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8c9c8-6369-44c4-943b-ea16f483444b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:36.106491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8c9c8-6369-44c4-943b-ea16f483444b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:36.106776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8c9c8-6369-44c4-943b-ea16f483444b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:36.107052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8c9c8-6369-44c4-943b-ea16f483444b None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:36.107775 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d8c9c8-6369-44c4-943b-ea16f483444b None admin] RBAC: Adding query filter params (name=network) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:36.108081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d8c9c8-6369-44c4-943b-ea16f483444b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:36.109509 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d8c9c8-6369-44c4-943b-ea16f483444b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:36.119386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 404/811] 10.222.0.65 () {56 vars in 1112 bytes} [Tue May 10 19:58:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:36.119386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769e66e-870d-4b9b-a723-d037b3267130 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:36.119386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769e66e-870d-4b9b-a723-d037b3267130 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:36.119386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769e66e-870d-4b9b-a723-d037b3267130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:36.137856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-3769e66e-870d-4b9b-a723-d037b3267130 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:36.139386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3769e66e-870d-4b9b-a723-d037b3267130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:36.198505 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81d33c-abdb-42d9-b9a5-2befc2c9a1d2 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:36.226103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 405/812] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:36.226103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769e66e-870d-4b9b-a723-d037b3267130 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:36.255634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8f1d9-7eaa-401a-a2bb-fbf934bad1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:36.256939 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8f1d9-7eaa-401a-a2bb-fbf934bad1e3 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:36.257919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8f1d9-7eaa-401a-a2bb-fbf934bad1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:36.258673 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8f1d9-7eaa-401a-a2bb-fbf934bad1e3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:36.268360 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8f1d9-7eaa-401a-a2bb-fbf934bad1e3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:36.270524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8f1d9-7eaa-401a-a2bb-fbf934bad1e3 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:36.290311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-8bc8f1d9-7eaa-401a-a2bb-fbf934bad1e3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 10 19:58:36.290311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:36.290311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:36.290311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:36.290311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:36.290311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:36.290311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:36.290311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:36.290311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:36.290311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:36.290311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:36.290311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:36.290311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:36.290311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:36.290311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:36.290311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:36.291518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:36.291518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:36.291518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:36.291518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:36.291518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:36.291518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:36.291518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:36.291518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:36.291518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:36.291518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:36.291518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:36.291518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:36.291518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:36.291518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:36.291518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:36.291518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:36.291518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:36.291518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:36.292721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:36.292721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:36.292721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:36.292721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:36.292721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:36.292721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:36.292721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:36.292721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:36.292721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 10 19:58:36.292721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:36.299509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 408/813] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:58:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:36.299509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8f1d9-7eaa-401a-a2bb-fbf934bad1e3 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:36.361349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769e66e-870d-4b9b-a723-d037b3267130 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:36.399322 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8f1d9-7eaa-401a-a2bb-fbf934bad1e3 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:36.406156 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6129c-f7db-4cd4-a2b0-b6cf667b3c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:36.407032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6129c-f7db-4cd4-a2b0-b6cf667b3c38 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:36.408446 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6129c-f7db-4cd4-a2b0-b6cf667b3c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:36.408919 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6129c-f7db-4cd4-a2b0-b6cf667b3c38 None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:36.409840 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6129c-f7db-4cd4-a2b0-b6cf667b3c38 None admin] RBAC: Adding query filter params (type=network) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:36.410661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6129c-f7db-4cd4-a2b0-b6cf667b3c38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:36.412812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6129c-f7db-4cd4-a2b0-b6cf667b3c38 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:36.426386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 406/814] 10.222.0.65 () {56 vars in 1112 bytes} [Tue May 10 19:58:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:36.426386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6129c-f7db-4cd4-a2b0-b6cf667b3c38 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:36.447268 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07dbe8e-d6b2-4e1f-8585-6f5223ce7331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:36.448261 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07dbe8e-d6b2-4e1f-8585-6f5223ce7331 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:36.448639 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07dbe8e-d6b2-4e1f-8585-6f5223ce7331 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:36.449028 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07dbe8e-d6b2-4e1f-8585-6f5223ce7331 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:36.462279 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07dbe8e-d6b2-4e1f-8585-6f5223ce7331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:36.473649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-a07dbe8e-d6b2-4e1f-8585-6f5223ce7331 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:36.473649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:36.473649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:36.473649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:36.473649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:36.473649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:36.473649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:36.473649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:36.473649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:36.473649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:36.473649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:36.473649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:36.473649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:36.473649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:36.473649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:36.473649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:36.474848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:36.474848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:36.474848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:36.474848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:36.474848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:36.474848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:36.474848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:36.474848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:36.474848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:36.474848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:36.474848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:36.474848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:36.474848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:36.474848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:36.474848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:36.474848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:36.475878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:36.475878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:36.475878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:36.475878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:36.475878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:36.475878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:36.475878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:36.475878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:36.475878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:36.475878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:36.475878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:36.475878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:36.482219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 409/815] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:36.482219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07dbe8e-d6b2-4e1f-8585-6f5223ce7331 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:36.562700 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6129c-f7db-4cd4-a2b0-b6cf667b3c38 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:36.587034 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07dbe8e-d6b2-4e1f-8585-6f5223ce7331 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:36.598567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90f099-67dd-4995-90fc-5c5e14a1fac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:36.599499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90f099-67dd-4995-90fc-5c5e14a1fac4 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:36.600026 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90f099-67dd-4995-90fc-5c5e14a1fac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:36.600486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90f099-67dd-4995-90fc-5c5e14a1fac4 None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:36.601518 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90f099-67dd-4995-90fc-5c5e14a1fac4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:36.602022 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90f099-67dd-4995-90fc-5c5e14a1fac4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:36.603692 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90f099-67dd-4995-90fc-5c5e14a1fac4 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:36.622511 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 407/816] 10.222.0.65 () {56 vars in 1135 bytes} [Tue May 10 19:58:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:36.623675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90f099-67dd-4995-90fc-5c5e14a1fac4 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:36.627155 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a607e14-6735-4220-8b20-cc43555886b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:36.628166 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a607e14-6735-4220-8b20-cc43555886b2 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:36.628166 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a607e14-6735-4220-8b20-cc43555886b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:36.628463 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a607e14-6735-4220-8b20-cc43555886b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:36.629228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a607e14-6735-4220-8b20-cc43555886b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:36.630679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a607e14-6735-4220-8b20-cc43555886b2 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:36.713534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 410/817] 10.222.0.65 () {60 vars in 1144 bytes} [Tue May 10 19:58:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:58:36.716117 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a607e14-6735-4220-8b20-cc43555886b2 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:36.752389 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90f099-67dd-4995-90fc-5c5e14a1fac4 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:36.843366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e508c6-ffeb-4ce8-be85-10be62edef53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:36.844442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e508c6-ffeb-4ce8-be85-10be62edef53 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:36.845876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e508c6-ffeb-4ce8-be85-10be62edef53 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:36.846416 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e508c6-ffeb-4ce8-be85-10be62edef53 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:36.848845 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e508c6-ffeb-4ce8-be85-10be62edef53 None admin] RBAC: Adding query filter params (name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:36.850650 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e508c6-ffeb-4ce8-be85-10be62edef53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:36.852336 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e508c6-ffeb-4ce8-be85-10be62edef53 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:36.871398 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 408/818] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 19:58:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:36.887825 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e508c6-ffeb-4ce8-be85-10be62edef53 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:36.937832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a607e14-6735-4220-8b20-cc43555886b2 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:36.975022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b9eaa-36ec-40b5-913a-6c201179f85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:36.975823 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b9eaa-36ec-40b5-913a-6c201179f85b None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:36.976171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b9eaa-36ec-40b5-913a-6c201179f85b None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:36.976368 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b9eaa-36ec-40b5-913a-6c201179f85b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:36.988081 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b9eaa-36ec-40b5-913a-6c201179f85b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:36.989792 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b9eaa-36ec-40b5-913a-6c201179f85b None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:37.007612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-e24b9eaa-36ec-40b5-913a-6c201179f85b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:37.007612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:37.007612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:37.007612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:37.007612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:37.007612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:37.007612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:37.007612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:37.007612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:37.007612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:37.007612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:37.007612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:37.007612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:37.007612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:37.007612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:37.008546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:37.008546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:37.008546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:37.008546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:37.008546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:37.008546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:37.008546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:37.008546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:37.008546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:37.008546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:37.008546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:37.008546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:37.008546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:37.008546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:37.008546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:37.008546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:37.008546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:37.009422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:37.009422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:37.009422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:37.009422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:37.013350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 411/819] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:37.014573 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b9eaa-36ec-40b5-913a-6c201179f85b None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:37.021823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e508c6-ffeb-4ce8-be85-10be62edef53 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:37.058043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d1e5f-fe32-4628-8183-814736745d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:37.059264 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d1e5f-fe32-4628-8183-814736745d55 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:37.059753 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d1e5f-fe32-4628-8183-814736745d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:37.060227 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d1e5f-fe32-4628-8183-814736745d55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:37.071793 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d1e5f-fe32-4628-8183-814736745d55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:37.073530 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d1e5f-fe32-4628-8183-814736745d55 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:37.092258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-912d1e5f-fe32-4628-8183-814736745d55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:37.092258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:37.092258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:37.092258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:37.092258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:37.092258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:37.092258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:37.092258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:37.092258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:37.092258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:37.092258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:37.092258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:37.092258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:37.092258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:37.092258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:37.093274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:37.093274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:37.093274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:37.093274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:37.093274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:37.093274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:37.093274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:37.093274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:37.093274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:37.093274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:37.093274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:37.093274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:37.093274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:37.093274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:37.093274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:37.093274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:37.093274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:37.094194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:37.094194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:37.094194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:37.094194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:37.096455 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 409/820] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:37.101625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d1e5f-fe32-4628-8183-814736745d55 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:37.138116 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b9eaa-36ec-40b5-913a-6c201179f85b None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:37.181127 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76268116-9779-4bde-872a-d880bdf7bac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:37.181900 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76268116-9779-4bde-872a-d880bdf7bac5 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:37.182548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76268116-9779-4bde-872a-d880bdf7bac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:37.182828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76268116-9779-4bde-872a-d880bdf7bac5 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:37.183883 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76268116-9779-4bde-872a-d880bdf7bac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:37.184196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76268116-9779-4bde-872a-d880bdf7bac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:37.186050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76268116-9779-4bde-872a-d880bdf7bac5 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:37.209181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 412/821] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:37.210841 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76268116-9779-4bde-872a-d880bdf7bac5 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:37.240955 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d1e5f-fe32-4628-8183-814736745d55 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:37.277348 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bde418-8a53-4c7a-9ca2-9db85fef8790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:37.278431 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bde418-8a53-4c7a-9ca2-9db85fef8790 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:37.278928 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bde418-8a53-4c7a-9ca2-9db85fef8790 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:37.279412 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bde418-8a53-4c7a-9ca2-9db85fef8790 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:37.280370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bde418-8a53-4c7a-9ca2-9db85fef8790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:37.280965 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bde418-8a53-4c7a-9ca2-9db85fef8790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:37.282645 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bde418-8a53-4c7a-9ca2-9db85fef8790 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:37.296166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 410/822] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:37.301855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bde418-8a53-4c7a-9ca2-9db85fef8790 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:37.327813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76268116-9779-4bde-872a-d880bdf7bac5 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:37.367878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b33d95-ca6f-4654-bba1-c4ca1b3f3b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:37.369058 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b33d95-ca6f-4654-bba1-c4ca1b3f3b74 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:37.369586 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b33d95-ca6f-4654-bba1-c4ca1b3f3b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:37.370059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b33d95-ca6f-4654-bba1-c4ca1b3f3b74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:37.377723 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b33d95-ca6f-4654-bba1-c4ca1b3f3b74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:37.379448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b33d95-ca6f-4654-bba1-c4ca1b3f3b74 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:37.393882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-a6b33d95-ca6f-4654-bba1-c4ca1b3f3b74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:37.393882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:37.393882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:37.393882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:37.393882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:37.393882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:37.393882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:37.393882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:37.393882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:37.393882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:37.393882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:37.393882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:37.393882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:37.393882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:37.393882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:37.394994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:37.394994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:37.394994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:37.394994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:37.394994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:37.394994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:37.394994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:37.394994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:37.394994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:37.394994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:37.394994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:37.394994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:37.394994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:37.394994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:37.394994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:37.394994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:37.394994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:37.395895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:37.395895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:37.395895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:37.395895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:37.395895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:37.395895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:37.395895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:37.395895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:37.402691 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 413/823] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:37.404756 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b33d95-ca6f-4654-bba1-c4ca1b3f3b74 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:37.410391 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bde418-8a53-4c7a-9ca2-9db85fef8790 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:37.450626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3da85-3c96-4f4d-93d6-4cb6fc331198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:37.453090 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3da85-3c96-4f4d-93d6-4cb6fc331198 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:37.454145 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3da85-3c96-4f4d-93d6-4cb6fc331198 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:37.454878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3da85-3c96-4f4d-93d6-4cb6fc331198 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:37.462614 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3da85-3c96-4f4d-93d6-4cb6fc331198 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:37.464357 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3da85-3c96-4f4d-93d6-4cb6fc331198 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:37.479935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-8bf3da85-3c96-4f4d-93d6-4cb6fc331198 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 19:58:37.479935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:37.479935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:37.479935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:37.479935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:37.479935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:37.479935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:37.479935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:37.479935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:37.479935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:37.479935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:37.479935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:37.479935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:37.479935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:37.479935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:37.479935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:37.480993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:37.480993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:37.480993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:37.480993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:37.480993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:37.480993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:37.480993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:37.480993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:37.480993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:37.480993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:37.480993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:37.480993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:37.480993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:37.480993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:37.480993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:37.480993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:37.480993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:37.480993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:37.481948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:37.481948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:37.481948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:37.481948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:37.481948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:37.481948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:37.481948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:37.481948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:37.481948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 19:58:37.481948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:37.484291 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 411/824] 10.222.0.65 () {56 vars in 1117 bytes} [Tue May 10 19:58:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:37.490177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3da85-3c96-4f4d-93d6-4cb6fc331198 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:37.536116 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b33d95-ca6f-4654-bba1-c4ca1b3f3b74 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:37.582101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9186a-8453-4d7b-964a-a35e7ae6f453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:37.582774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9186a-8453-4d7b-964a-a35e7ae6f453 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:37.583057 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9186a-8453-4d7b-964a-a35e7ae6f453 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:37.583330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9186a-8453-4d7b-964a-a35e7ae6f453 None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:37.584312 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9186a-8453-4d7b-964a-a35e7ae6f453 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:37.584930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9186a-8453-4d7b-964a-a35e7ae6f453 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:37.586814 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9186a-8453-4d7b-964a-a35e7ae6f453 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:37.593111 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3da85-3c96-4f4d-93d6-4cb6fc331198 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:37.606126 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 414/825] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:37.609753 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9186a-8453-4d7b-964a-a35e7ae6f453 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:37.635763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734a293-14a4-4632-9673-ab9ef243929f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:37.636433 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734a293-14a4-4632-9673-ab9ef243929f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:37.636719 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734a293-14a4-4632-9673-ab9ef243929f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:37.636994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734a293-14a4-4632-9673-ab9ef243929f None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:37.638557 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734a293-14a4-4632-9673-ab9ef243929f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:37.639012 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734a293-14a4-4632-9673-ab9ef243929f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:37.641151 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734a293-14a4-4632-9673-ab9ef243929f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:37.667442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 412/826] 10.222.0.65 () {56 vars in 1149 bytes} [Tue May 10 19:58:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:37.668850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734a293-14a4-4632-9673-ab9ef243929f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:37.738230 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9186a-8453-4d7b-964a-a35e7ae6f453 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:37.779755 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c6fa3-e0de-480d-b2b2-a4d71ad96179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:37.780809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c6fa3-e0de-480d-b2b2-a4d71ad96179 None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:37.781094 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c6fa3-e0de-480d-b2b2-a4d71ad96179 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:37.781367 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c6fa3-e0de-480d-b2b2-a4d71ad96179 None admin] PATH_INFO: `/v3/projects/a006477d1df94390b599621446bd0f02/users/b4d10b258a53484da22950e2b1db2e41/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:37.785990 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c6fa3-e0de-480d-b2b2-a4d71ad96179 None admin] RBAC: Authorizing `identity:create_grant(project_id=a006477d1df94390b599621446bd0f02, user_id=b4d10b258a53484da22950e2b1db2e41, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:37.787479 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c6fa3-e0de-480d-b2b2-a4d71ad96179 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:37.795992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734a293-14a4-4632-9673-ab9ef243929f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:37.829420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24c7dd-408c-4b28-8fb1-a2c9302b7964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:37.830295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24c7dd-408c-4b28-8fb1-a2c9302b7964 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:37.830580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24c7dd-408c-4b28-8fb1-a2c9302b7964 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:37.830901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24c7dd-408c-4b28-8fb1-a2c9302b7964 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:37.840923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24c7dd-408c-4b28-8fb1-a2c9302b7964 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:37.842385 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24c7dd-408c-4b28-8fb1-a2c9302b7964 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:37.860945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-3e24c7dd-408c-4b28-8fb1-a2c9302b7964 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:37.860945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:37.860945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:37.860945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:37.860945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:37.860945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:37.860945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:37.860945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:37.860945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:37.860945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:37.860945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:37.860945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:37.860945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:37.860945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:37.860945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:37.861912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:37.861912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:37.861912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:37.861912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:37.861912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:37.861912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:37.861912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:37.861912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:37.861912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:37.861912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:37.861912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:37.861912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:37.861912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:37.861912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:37.861912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:37.861912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:37.861912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:37.862944 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:37.862944 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:37.862944 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:37.862944 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:37.862944 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 413/827] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:37.868836 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24c7dd-408c-4b28-8fb1-a2c9302b7964 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:37.881378 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 415/828] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:58:37 2022] PUT /identity/v3/projects/a006477d1df94390b599621446bd0f02/users/b4d10b258a53484da22950e2b1db2e41/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 19:58:37.900533 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613e5c5-7463-4737-9acf-ff9e943ad6c3 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:37.901189 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613e5c5-7463-4737-9acf-ff9e943ad6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:37.901674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613e5c5-7463-4737-9acf-ff9e943ad6c3 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:37.903608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 416/829] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:58:37.911541 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb1bab-c271-4313-82bb-c05573f9e0e7 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:37.911541 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb1bab-c271-4313-82bb-c05573f9e0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:37.911812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb1bab-c271-4313-82bb-c05573f9e0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:37.932706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-43cb1bab-c271-4313-82bb-c05573f9e0e7 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:37.933951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-43cb1bab-c271-4313-82bb-c05573f9e0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:37.999634 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24c7dd-408c-4b28-8fb1-a2c9302b7964 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:38.018466 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 417/830] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:58:38.036789 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb1bab-c271-4313-82bb-c05573f9e0e7 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:38.039168 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa011be-d789-446c-9bc8-56e1c9572b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:38.040145 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa011be-d789-446c-9bc8-56e1c9572b60 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:38.040664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa011be-d789-446c-9bc8-56e1c9572b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:38.041193 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa011be-d789-446c-9bc8-56e1c9572b60 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:38.042208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa011be-d789-446c-9bc8-56e1c9572b60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:38.042777 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa011be-d789-446c-9bc8-56e1c9572b60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:38.044528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa011be-d789-446c-9bc8-56e1c9572b60 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:38.057863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 414/831] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:38.064075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa011be-d789-446c-9bc8-56e1c9572b60 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:38.146869 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb1bab-c271-4313-82bb-c05573f9e0e7 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:38.175613 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa011be-d789-446c-9bc8-56e1c9572b60 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:38.180801 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e671e35-4334-477e-9739-5e214588fe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:38.181612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e671e35-4334-477e-9739-5e214588fe62 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:38.181897 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e671e35-4334-477e-9739-5e214588fe62 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:38.182204 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e671e35-4334-477e-9739-5e214588fe62 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:38.182920 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e671e35-4334-477e-9739-5e214588fe62 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:38.184240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e671e35-4334-477e-9739-5e214588fe62 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:38.196412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-9e671e35-4334-477e-9739-5e214588fe62 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 19:58:38.196412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:38.196412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:38.196412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:38.196412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:38.196412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:38.196412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:38.196412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:38.196412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 19:58:38.196412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:38.196412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:38.196412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:38.196412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:38.196412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:38.196412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:38.196412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:38.197349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:38.197349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:38.197349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:38.197349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:38.197349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:38.197349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:38.197349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:38.197349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:38.197349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:38.197349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:38.197349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:38.197349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:38.197349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:38.197349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:38.197349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:38.197349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:38.198233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:38.198233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:38.198233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:38.198233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:38.198233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:38.198233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:38.198233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:38.198233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:38.198233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:38.198233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:38.198233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:38.198233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:38.198233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:38.198233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 19:58:38.198233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:38.198233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 418/832] 10.222.0.65 () {56 vars in 1132 bytes} [Tue May 10 19:58:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:38.199084 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50c667-4ed9-4669-9c8c-582fd36c13c6 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:38.199084 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50c667-4ed9-4669-9c8c-582fd36c13c6 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:38.199765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50c667-4ed9-4669-9c8c-582fd36c13c6 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:38.203207 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 419/833] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:38.206199 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50c667-4ed9-4669-9c8c-582fd36c13c6 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:38.215151 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7772845-8548-45e7-b5ec-18f808be4b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:38.216201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7772845-8548-45e7-b5ec-18f808be4b8b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:38.216201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7772845-8548-45e7-b5ec-18f808be4b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:38.216475 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7772845-8548-45e7-b5ec-18f808be4b8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:38.224147 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7772845-8548-45e7-b5ec-18f808be4b8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:38.225596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7772845-8548-45e7-b5ec-18f808be4b8b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:38.240016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-d7772845-8548-45e7-b5ec-18f808be4b8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:38.240016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:38.240016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:38.240016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:38.240016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:38.240016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:38.240016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:38.240016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:38.240016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:38.240016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:38.240016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:38.240016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:38.240016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:38.240016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:38.240016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:38.240898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:38.240898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:38.240898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:38.240898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:38.240898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:38.240898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:38.240898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:38.240898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:38.240898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:38.240898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:38.240898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:38.240898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:38.240898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:38.240898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:38.240898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:38.240898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:38.240898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:38.241781 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:38.241781 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:38.241781 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:38.241781 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:38.241781 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:38.241781 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:38.241781 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:38.241781 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:38.241781 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 415/834] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:38.242931 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cba59-8898-4faa-bd0f-daaf33da25ae None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:38.242931 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cba59-8898-4faa-bd0f-daaf33da25ae None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:38.242931 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cba59-8898-4faa-bd0f-daaf33da25ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:38.259390 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-d26cba59-8898-4faa-bd0f-daaf33da25ae None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:38.260582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-d26cba59-8898-4faa-bd0f-daaf33da25ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:38.315498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50c667-4ed9-4669-9c8c-582fd36c13c6 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:38.340277 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 416/835] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:38.340277 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26cba59-8898-4faa-bd0f-daaf33da25ae None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:38.350065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920acc59-e60b-44ce-950e-7595661dba2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:38.350774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920acc59-e60b-44ce-950e-7595661dba2a None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:38.351103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920acc59-e60b-44ce-950e-7595661dba2a None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:38.351376 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920acc59-e60b-44ce-950e-7595661dba2a None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:38.352148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920acc59-e60b-44ce-950e-7595661dba2a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:38.352455 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920acc59-e60b-44ce-950e-7595661dba2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:38.353937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920acc59-e60b-44ce-950e-7595661dba2a None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:38.359295 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 420/836] 10.222.0.65 () {56 vars in 1126 bytes} [Tue May 10 19:58:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:38.359847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920acc59-e60b-44ce-950e-7595661dba2a None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:38.447488 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26cba59-8898-4faa-bd0f-daaf33da25ae None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:38.464560 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920acc59-e60b-44ce-950e-7595661dba2a None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:38.480415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae512b-edba-43a3-afa6-a280783236bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:38.481070 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae512b-edba-43a3-afa6-a280783236bd None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:38.481350 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae512b-edba-43a3-afa6-a280783236bd None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:38.481621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae512b-edba-43a3-afa6-a280783236bd None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:38.482437 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae512b-edba-43a3-afa6-a280783236bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:38.482743 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae512b-edba-43a3-afa6-a280783236bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:38.484160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae512b-edba-43a3-afa6-a280783236bd None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:38.500192 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 417/837] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:38.500192 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae512b-edba-43a3-afa6-a280783236bd None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:38.503095 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de997f0c-f198-4e9e-9753-556057cef0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:38.503891 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de997f0c-f198-4e9e-9753-556057cef0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:38.504175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de997f0c-f198-4e9e-9753-556057cef0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:38.504446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de997f0c-f198-4e9e-9753-556057cef0a0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:38.505111 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de997f0c-f198-4e9e-9753-556057cef0a0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:38.506927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de997f0c-f198-4e9e-9753-556057cef0a0 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:38.519854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-de997f0c-f198-4e9e-9753-556057cef0a0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 19:58:38.519854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:38.519854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:38.519854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:38.519854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:38.519854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:38.519854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:38.519854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:38.519854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 19:58:38.519854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:38.519854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:38.519854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:38.519854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:38.519854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:38.519854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:38.519854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:38.519854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:38.520846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:38.520846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:38.520846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:38.520846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:38.520846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:38.520846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:38.520846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:38.520846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:38.520846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:38.520846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:38.520846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:38.520846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:38.520846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:38.520846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:38.520846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:38.520846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:38.521670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:38.521670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:38.521670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:38.521670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:38.521670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:38.521670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:38.521670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:38.521670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:38.521670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:38.521670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:38.521670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:38.521670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:38.521670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 19:58:38.521670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:38.521670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 421/838] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:58:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:38.522534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de997f0c-f198-4e9e-9753-556057cef0a0 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:38.599986 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae512b-edba-43a3-afa6-a280783236bd None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:38.621657 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de997f0c-f198-4e9e-9753-556057cef0a0 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:38.633240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bcdd1d-3c8d-4893-a67b-ca37abfbd52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:38.633909 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcdd1d-3c8d-4893-a67b-ca37abfbd52c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:38.634224 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcdd1d-3c8d-4893-a67b-ca37abfbd52c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:38.634501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcdd1d-3c8d-4893-a67b-ca37abfbd52c None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:38.635216 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bcdd1d-3c8d-4893-a67b-ca37abfbd52c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:38.635525 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bcdd1d-3c8d-4893-a67b-ca37abfbd52c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:38.636878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bcdd1d-3c8d-4893-a67b-ca37abfbd52c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:38.641240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 418/839] 10.222.0.65 () {56 vars in 1126 bytes} [Tue May 10 19:58:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:38.642604 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bcdd1d-3c8d-4893-a67b-ca37abfbd52c None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:38.666875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1efe5-efd3-4e3d-b935-3c65b0b3e247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:38.668113 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1efe5-efd3-4e3d-b935-3c65b0b3e247 None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:38.668625 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1efe5-efd3-4e3d-b935-3c65b0b3e247 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:38.669076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1efe5-efd3-4e3d-b935-3c65b0b3e247 None admin] PATH_INFO: `/v3/projects/a006477d1df94390b599621446bd0f02/users/7ec2605fd1b84e0ea134b943ed33a605/roles/55528a93cfa64162b2f9574fb5cdde79` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:38.689546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1efe5-efd3-4e3d-b935-3c65b0b3e247 None admin] RBAC: Authorizing `identity:create_grant(project_id=a006477d1df94390b599621446bd0f02, user_id=7ec2605fd1b84e0ea134b943ed33a605, role_id=55528a93cfa64162b2f9574fb5cdde79)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:38.691057 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1efe5-efd3-4e3d-b935-3c65b0b3e247 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:38.759502 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bcdd1d-3c8d-4893-a67b-ca37abfbd52c None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:38.775020 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 422/840] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:58:38 2022] PUT /identity/v3/projects/a006477d1df94390b599621446bd0f02/users/7ec2605fd1b84e0ea134b943ed33a605/roles/55528a93cfa64162b2f9574fb5cdde79 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:58:38.805353 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff537c-aad3-490b-83f3-9fac86c05b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:38.806149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff537c-aad3-490b-83f3-9fac86c05b3c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:38.806451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff537c-aad3-490b-83f3-9fac86c05b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:38.806797 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff537c-aad3-490b-83f3-9fac86c05b3c None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:38.807588 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff537c-aad3-490b-83f3-9fac86c05b3c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:38.807908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff537c-aad3-490b-83f3-9fac86c05b3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:38.809671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff537c-aad3-490b-83f3-9fac86c05b3c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:38.816249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 419/841] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:58:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:38.822463 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1efe5-efd3-4e3d-b935-3c65b0b3e247 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:38.938118 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1efe5-efd3-4e3d-b935-3c65b0b3e247 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:38.974860 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6241f4-38d2-486a-bf16-30c999c6d735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:38.975530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6241f4-38d2-486a-bf16-30c999c6d735 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:38.975819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6241f4-38d2-486a-bf16-30c999c6d735 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:38.976096 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6241f4-38d2-486a-bf16-30c999c6d735 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:38.976819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6241f4-38d2-486a-bf16-30c999c6d735 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:38.977126 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6241f4-38d2-486a-bf16-30c999c6d735 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:38.978543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6241f4-38d2-486a-bf16-30c999c6d735 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:38.983333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 423/842] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:58:38 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:40.173262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8683f-b1ab-4d1a-91b9-c7d80d99e6f0 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:40.173772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8683f-b1ab-4d1a-91b9-c7d80d99e6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:40.173850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8683f-b1ab-4d1a-91b9-c7d80d99e6f0 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:40.175428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 420/843] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:40.180984 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12644475-13f9-49d9-a85f-db6fad34972a None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:40.181388 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12644475-13f9-49d9-a85f-db6fad34972a None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:40.181589 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12644475-13f9-49d9-a85f-db6fad34972a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:40.198835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-12644475-13f9-49d9-a85f-db6fad34972a None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:40.200016 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-12644475-13f9-49d9-a85f-db6fad34972a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:40.268135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 424/844] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:40.290884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8683f-b1ab-4d1a-91b9-c7d80d99e6f0 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:40.393920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8683f-b1ab-4d1a-91b9-c7d80d99e6f0 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:40.429266 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bb3e2-3043-40ca-a1c2-09503b52b783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:40.429266 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bb3e2-3043-40ca-a1c2-09503b52b783 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:40.429266 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bb3e2-3043-40ca-a1c2-09503b52b783 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:40.429266 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bb3e2-3043-40ca-a1c2-09503b52b783 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:40.440164 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753bb3e2-3043-40ca-a1c2-09503b52b783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:40.447728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-753bb3e2-3043-40ca-a1c2-09503b52b783 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:58:40.447728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:40.447728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:40.447728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:40.447728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:40.447728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:40.447728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:40.447728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:40.447728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:40.447728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:40.447728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:40.447728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:40.447728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:40.447728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:40.447728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:40.447728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:40.448653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:40.448653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:40.448653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:40.448653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:40.448653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:40.448653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:40.448653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:40.448653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:40.448653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:40.448653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:40.448653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:40.448653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:40.448653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:40.448653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:40.448653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:40.448653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:40.450423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:40.450423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:40.450423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:40.450423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:40.450423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:40.450423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:40.450423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:40.450423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:40.450423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:40.450423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:40.450423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:58:40.450423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:40.450423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 421/845] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:58:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:40.455047 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12644475-13f9-49d9-a85f-db6fad34972a None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:40.560337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12644475-13f9-49d9-a85f-db6fad34972a None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:40.596271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a229fcb-87cf-42da-b38e-f7acc5022499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:40.596941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a229fcb-87cf-42da-b38e-f7acc5022499 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:40.597220 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a229fcb-87cf-42da-b38e-f7acc5022499 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:40.597494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a229fcb-87cf-42da-b38e-f7acc5022499 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:40.598275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a229fcb-87cf-42da-b38e-f7acc5022499 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:40.598584 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a229fcb-87cf-42da-b38e-f7acc5022499 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:40.599943 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a229fcb-87cf-42da-b38e-f7acc5022499 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:40.608803 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 425/846] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:58:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:40.615015 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bb3e2-3043-40ca-a1c2-09503b52b783 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:40.718535 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bb3e2-3043-40ca-a1c2-09503b52b783 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:40.752847 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d111c-eba1-4db4-b90f-4bd3c8be87e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:40.753637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d111c-eba1-4db4-b90f-4bd3c8be87e7 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:40.753918 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d111c-eba1-4db4-b90f-4bd3c8be87e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:40.754218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d111c-eba1-4db4-b90f-4bd3c8be87e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:40.765253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d111c-eba1-4db4-b90f-4bd3c8be87e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:40.766771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d111c-eba1-4db4-b90f-4bd3c8be87e7 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:40.784691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-0d6d111c-eba1-4db4-b90f-4bd3c8be87e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:40.784691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:40.784691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:40.784691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:40.784691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:40.784691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:40.784691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:40.784691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:40.784691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:40.784691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:40.784691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:40.784691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:40.784691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:40.784691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:40.784691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:40.787010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:40.787010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:40.787010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:40.787010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:40.787010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:40.787010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:40.787010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:40.787010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:40.787010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:40.787010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:40.787010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:40.787010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:40.787010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:40.787010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:40.787010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:40.787010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:40.787010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:40.789127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:40.789127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:40.789127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:40.789127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:40.789127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 422/847] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:40.792105 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a229fcb-87cf-42da-b38e-f7acc5022499 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:40.908562 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a229fcb-87cf-42da-b38e-f7acc5022499 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:40.942854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a6f98-7cf7-4609-b1f2-eafb6541d08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:40.943513 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a6f98-7cf7-4609-b1f2-eafb6541d08b None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:40.943797 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a6f98-7cf7-4609-b1f2-eafb6541d08b None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:40.944073 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a6f98-7cf7-4609-b1f2-eafb6541d08b None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:40.944806 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a6f98-7cf7-4609-b1f2-eafb6541d08b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:40.945109 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a6f98-7cf7-4609-b1f2-eafb6541d08b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:40.946519 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a6f98-7cf7-4609-b1f2-eafb6541d08b None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:40.959538 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 426/848] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:40.967173 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d111c-eba1-4db4-b90f-4bd3c8be87e7 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:41.002366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db23a2-21e8-4828-894b-91268f0d0530 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:41.002657 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db23a2-21e8-4828-894b-91268f0d0530 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:41.002851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db23a2-21e8-4828-894b-91268f0d0530 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:41.005249 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 427/849] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:58:41.011063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a515a-dbc9-46f6-a583-db9826041b7c None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:41.011673 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a515a-dbc9-46f6-a583-db9826041b7c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:41.011955 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a515a-dbc9-46f6-a583-db9826041b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:41.032916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-502a515a-dbc9-46f6-a583-db9826041b7c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:41.034130 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-502a515a-dbc9-46f6-a583-db9826041b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:41.074564 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d111c-eba1-4db4-b90f-4bd3c8be87e7 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:41.104960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 428/850] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:58:41.106612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8372d-4de7-4234-93ad-87b4a5972078 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:41.106612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8372d-4de7-4234-93ad-87b4a5972078 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:41.106869 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8372d-4de7-4234-93ad-87b4a5972078 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:41.112033 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 429/851] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:41.113582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf469595-920c-48d3-8165-c07732a0fda1 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:41.113826 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf469595-920c-48d3-8165-c07732a0fda1 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:41.114128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf469595-920c-48d3-8165-c07732a0fda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:41.118721 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5a291-42cc-42a6-b926-537030daa6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:41.120103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5a291-42cc-42a6-b926-537030daa6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:41.120389 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5a291-42cc-42a6-b926-537030daa6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:41.120657 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5a291-42cc-42a6-b926-537030daa6b1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:41.128733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5a291-42cc-42a6-b926-537030daa6b1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:41.130394 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5a291-42cc-42a6-b926-537030daa6b1 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:41.130983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-bf469595-920c-48d3-8165-c07732a0fda1 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:41.132136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-bf469595-920c-48d3-8165-c07732a0fda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:41.144905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-33e5a291-42cc-42a6-b926-537030daa6b1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 10 19:58:41.144905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:41.144905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:41.144905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:41.144905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:41.144905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:41.144905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:41.144905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:41.144905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:41.144905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:41.144905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:41.144905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:41.144905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:41.144905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:41.144905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:41.144905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:41.145876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:41.145876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:41.145876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:41.145876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:41.145876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:41.145876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:41.145876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:41.145876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:41.145876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:41.145876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:41.145876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:41.145876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:41.145876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:41.145876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:41.145876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:41.145876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:41.145876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:41.145876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:41.146871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:41.146871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:41.146871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:41.146871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:41.146871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:41.146871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:41.146871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:41.146871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:41.146871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 10 19:58:41.146871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:41.146871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 423/852] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:58:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:41.146871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5a291-42cc-42a6-b926-537030daa6b1 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:41.206660 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 430/853] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:58:41.207505 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf469595-920c-48d3-8165-c07732a0fda1 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:41.265496 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5a291-42cc-42a6-b926-537030daa6b1 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:41.287493 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3a9b9-eabb-47b3-af8d-caf4890519df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:41.288150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3a9b9-eabb-47b3-af8d-caf4890519df None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:41.288434 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3a9b9-eabb-47b3-af8d-caf4890519df None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:41.288708 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3a9b9-eabb-47b3-af8d-caf4890519df None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:41.289430 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3a9b9-eabb-47b3-af8d-caf4890519df None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:41.290823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3a9b9-eabb-47b3-af8d-caf4890519df None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:41.311283 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf469595-920c-48d3-8165-c07732a0fda1 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:41.343290 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232a1de-a632-440b-822a-8134de9d45bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:41.343943 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232a1de-a632-440b-822a-8134de9d45bc None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:41.344225 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232a1de-a632-440b-822a-8134de9d45bc None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:41.345104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232a1de-a632-440b-822a-8134de9d45bc None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:41.345943 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232a1de-a632-440b-822a-8134de9d45bc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:41.346306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232a1de-a632-440b-822a-8134de9d45bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:41.347697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232a1de-a632-440b-822a-8134de9d45bc None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:41.364135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 431/854] 10.222.0.65 () {56 vars in 1135 bytes} [Tue May 10 19:58:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:41.364987 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232a1de-a632-440b-822a-8134de9d45bc None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:41.376243 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 424/855] 10.222.0.65 () {60 vars in 1141 bytes} [Tue May 10 19:58:41 2022] POST /identity/v3/services => generated 269 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:58:41.377917 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa8b56-85dc-469d-b35e-269f7581ea90 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:41.378249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa8b56-85dc-469d-b35e-269f7581ea90 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:41.378533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa8b56-85dc-469d-b35e-269f7581ea90 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:41.379932 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 425/856] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:41.380544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa8b56-85dc-469d-b35e-269f7581ea90 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:41.477313 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232a1de-a632-440b-822a-8134de9d45bc None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:41.508563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa8b56-85dc-469d-b35e-269f7581ea90 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:41.514985 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5d404-7263-4a64-8dcc-bf24d261ee1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:41.515857 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5d404-7263-4a64-8dcc-bf24d261ee1d None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:41.517274 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5d404-7263-4a64-8dcc-bf24d261ee1d None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:41.517274 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5d404-7263-4a64-8dcc-bf24d261ee1d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:41.543435 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89c5a1-f9c0-418a-b06f-41c09578d580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:41.544097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89c5a1-f9c0-418a-b06f-41c09578d580 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:41.544380 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89c5a1-f9c0-418a-b06f-41c09578d580 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:41.544654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89c5a1-f9c0-418a-b06f-41c09578d580 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:41.547145 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5d404-7263-4a64-8dcc-bf24d261ee1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:41.560091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-14c5d404-7263-4a64-8dcc-bf24d261ee1d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:41.560091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:41.560091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:41.560091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:41.560091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:41.560091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:41.560091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:41.560091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:41.560091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:41.560091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:41.560091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:41.560091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:41.560091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:41.560091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:41.560091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:41.560091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:41.565235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:41.565235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:41.565235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:41.565235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:41.565235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:41.565235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:41.565235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:41.565235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:41.565235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:41.565235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:41.565235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:41.565235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:41.565235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:41.565235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:41.565235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:41.565235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:41.566738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:41.566738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:41.566738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:41.566738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:41.566738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:41.566738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:41.566738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:41.566738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:41.566738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:41.566738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:41.566738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 19:58:41.566738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:41.566738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 432/857] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 19:58:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:41.568675 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979abc8-98cf-47d6-b9e9-4fc56436f004 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:41.568675 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979abc8-98cf-47d6-b9e9-4fc56436f004 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:41.568675 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979abc8-98cf-47d6-b9e9-4fc56436f004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:41.590381 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-0979abc8-98cf-47d6-b9e9-4fc56436f004 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:41.591844 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0979abc8-98cf-47d6-b9e9-4fc56436f004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:41.620812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89c5a1-f9c0-418a-b06f-41c09578d580 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:41.622290 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89c5a1-f9c0-418a-b06f-41c09578d580 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:41.662122 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 433/858] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:41 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:41.662982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979abc8-98cf-47d6-b9e9-4fc56436f004 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:41.728082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 426/859] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:58:41 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 19:58:41.729253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89c5a1-f9c0-418a-b06f-41c09578d580 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:41.760856 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979abc8-98cf-47d6-b9e9-4fc56436f004 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:41.794154 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eba27e-0050-4708-8f27-5a9b655229de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:41.794816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eba27e-0050-4708-8f27-5a9b655229de None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:41.795095 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eba27e-0050-4708-8f27-5a9b655229de None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:41.795366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eba27e-0050-4708-8f27-5a9b655229de None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:41.796111 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eba27e-0050-4708-8f27-5a9b655229de None admin] RBAC: Adding query filter params (name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:41.796413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eba27e-0050-4708-8f27-5a9b655229de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:41.798319 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eba27e-0050-4708-8f27-5a9b655229de None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:41.811250 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 434/860] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 19:58:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:41.811250 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eba27e-0050-4708-8f27-5a9b655229de None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:41.833942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89c5a1-f9c0-418a-b06f-41c09578d580 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:41.868025 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286c8d9-170e-43f2-9deb-d62cadac7fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:41.868684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286c8d9-170e-43f2-9deb-d62cadac7fd5 None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:41.869103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286c8d9-170e-43f2-9deb-d62cadac7fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:41.869378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286c8d9-170e-43f2-9deb-d62cadac7fd5 None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:41.870296 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0286c8d9-170e-43f2-9deb-d62cadac7fd5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:41.872139 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0286c8d9-170e-43f2-9deb-d62cadac7fd5 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:41.906058 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eba27e-0050-4708-8f27-5a9b655229de None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:41.942113 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5958d-8f7b-45f3-b91b-51b964049105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:41.942915 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5958d-8f7b-45f3-b91b-51b964049105 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:41.943198 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5958d-8f7b-45f3-b91b-51b964049105 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:41.943474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5958d-8f7b-45f3-b91b-51b964049105 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:41.953947 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5958d-8f7b-45f3-b91b-51b964049105 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:41.955431 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5958d-8f7b-45f3-b91b-51b964049105 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:41.970267 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 427/861] 10.222.0.65 () {60 vars in 1141 bytes} [Tue May 10 19:58:41 2022] POST /identity/v3/services => generated 251 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:58:41.970267 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286c8d9-170e-43f2-9deb-d62cadac7fd5 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:41.986525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-eec5958d-8f7b-45f3-b91b-51b964049105 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:41.986525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:41.986525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:41.986525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:41.986525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:41.986525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:41.986525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:41.986525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:41.986525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:41.986525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:41.986525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:41.986525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:41.986525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:41.986525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:41.986525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:41.987696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:41.987696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:41.987696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:41.987696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:41.987696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:41.987696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:41.987696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:41.987696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:41.987696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:41.987696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:41.987696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:41.987696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:41.987696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:41.987696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:41.987696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:41.987696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:41.987696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:41.989055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:41.989055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:41.989055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:41.989055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:41.994816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 435/862] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:41.995821 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5958d-8f7b-45f3-b91b-51b964049105 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:42.098670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286c8d9-170e-43f2-9deb-d62cadac7fd5 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:42.109255 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5958d-8f7b-45f3-b91b-51b964049105 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:42.145589 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2953c-1d1e-4a1f-b8e5-c1e8a7368487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:42.146317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08488d11-7d66-444b-ac28-f40f41e9ebe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:42.146705 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2953c-1d1e-4a1f-b8e5-c1e8a7368487 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:42.147066 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2953c-1d1e-4a1f-b8e5-c1e8a7368487 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:42.147418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2953c-1d1e-4a1f-b8e5-c1e8a7368487 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:42.147822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08488d11-7d66-444b-ac28-f40f41e9ebe8 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:42.148488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08488d11-7d66-444b-ac28-f40f41e9ebe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:42.148943 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08488d11-7d66-444b-ac28-f40f41e9ebe8 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:42.149871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08488d11-7d66-444b-ac28-f40f41e9ebe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:42.150410 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08488d11-7d66-444b-ac28-f40f41e9ebe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:42.152713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08488d11-7d66-444b-ac28-f40f41e9ebe8 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:42.163176 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b2953c-1d1e-4a1f-b8e5-c1e8a7368487 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:42.164936 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b2953c-1d1e-4a1f-b8e5-c1e8a7368487 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:42.168880 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 436/863] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:42.174466 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08488d11-7d66-444b-ac28-f40f41e9ebe8 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:42.185738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-01b2953c-1d1e-4a1f-b8e5-c1e8a7368487 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:42.185738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:42.185738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:42.185738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:42.185738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:42.185738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:42.185738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:42.185738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:42.185738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:42.185738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:42.185738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:42.185738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:42.185738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:42.185738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:42.185738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:42.186806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:42.186806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:42.186806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:42.186806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:42.186806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:42.186806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:42.186806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:42.186806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:42.186806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:42.186806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:42.186806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:42.186806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:42.186806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:42.186806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:42.186806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:42.186806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:42.186806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:42.188087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:42.188087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:42.188087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:42.188087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:42.188087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 428/864] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:42.192613 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2953c-1d1e-4a1f-b8e5-c1e8a7368487 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:42.291744 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08488d11-7d66-444b-ac28-f40f41e9ebe8 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:42.301309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2953c-1d1e-4a1f-b8e5-c1e8a7368487 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:42.328305 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003318e3-15c0-4b61-bb6f-4b46da9aa0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:42.329446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003318e3-15c0-4b61-bb6f-4b46da9aa0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:42.329952 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003318e3-15c0-4b61-bb6f-4b46da9aa0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:42.330452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003318e3-15c0-4b61-bb6f-4b46da9aa0a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:42.335751 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25abfe-6883-4a60-a715-f1595b71cbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:42.336614 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25abfe-6883-4a60-a715-f1595b71cbd4 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:42.336910 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25abfe-6883-4a60-a715-f1595b71cbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:42.337189 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25abfe-6883-4a60-a715-f1595b71cbd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:42.337939 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25abfe-6883-4a60-a715-f1595b71cbd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:42.338322 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25abfe-6883-4a60-a715-f1595b71cbd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:42.339189 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003318e3-15c0-4b61-bb6f-4b46da9aa0a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:42.339657 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25abfe-6883-4a60-a715-f1595b71cbd4 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:42.341035 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003318e3-15c0-4b61-bb6f-4b46da9aa0a9 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:42.352904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 429/865] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:42.355930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-003318e3-15c0-4b61-bb6f-4b46da9aa0a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:42.355930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:42.355930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:42.355930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:42.355930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:42.355930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:42.355930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:42.355930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:42.355930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:42.355930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:42.355930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:42.355930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:42.355930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:42.355930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:42.355930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:42.356783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:42.356783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:42.356783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:42.356783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:42.356783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:42.356783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:42.356783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:42.356783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:42.356783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:42.356783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:42.356783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:42.356783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:42.356783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:42.356783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:42.356783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:42.356783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:42.356783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:42.357666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:42.357666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:42.357666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:42.357666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:42.357666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:42.357666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:42.357666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:42.357666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:42.358556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25abfe-6883-4a60-a715-f1595b71cbd4 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:42.360841 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 437/866] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:42.366796 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003318e3-15c0-4b61-bb6f-4b46da9aa0a9 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:42.461369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25abfe-6883-4a60-a715-f1595b71cbd4 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:42.468167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003318e3-15c0-4b61-bb6f-4b46da9aa0a9 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:42.496622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1e31a-4a3b-4a16-9d91-7b39d45ae167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:42.497439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1e31a-4a3b-4a16-9d91-7b39d45ae167 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:42.497725 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1e31a-4a3b-4a16-9d91-7b39d45ae167 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:42.498000 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1e31a-4a3b-4a16-9d91-7b39d45ae167 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:42.504029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc86de-7aef-4122-85da-ef448195c025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:42.504949 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc86de-7aef-4122-85da-ef448195c025 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:42.505450 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc86de-7aef-4122-85da-ef448195c025 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:42.505556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1e31a-4a3b-4a16-9d91-7b39d45ae167 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:42.506070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc86de-7aef-4122-85da-ef448195c025 None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:42.507102 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc86de-7aef-4122-85da-ef448195c025 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:42.507306 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1e31a-4a3b-4a16-9d91-7b39d45ae167 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:42.507811 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc86de-7aef-4122-85da-ef448195c025 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:42.509410 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc86de-7aef-4122-85da-ef448195c025 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:42.528427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-ffa1e31a-4a3b-4a16-9d91-7b39d45ae167 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 19:58:42.528427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:42.528427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:42.528427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:42.528427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:42.528427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:42.528427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:42.528427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:42.528427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:42.528427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:42.528427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:42.528427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:42.528427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:42.528427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:42.528427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:42.528427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:42.529429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:42.529429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:42.529429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:42.529429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:42.529429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:42.529429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:42.529429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:42.529429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:42.529429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:42.529429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:42.529429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:42.529429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:42.529429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:42.529429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:42.529429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:42.529429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:42.529429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:42.529429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:42.530423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:42.530423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:42.530423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:42.530423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:42.530423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:42.530423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:42.530423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:42.530423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:42.530423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 19:58:42.530423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:42.530423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 430/868] 10.222.0.65 () {56 vars in 1117 bytes} [Tue May 10 19:58:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:42.531662 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1e31a-4a3b-4a16-9d91-7b39d45ae167 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:42.532494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 438/868] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:42.533664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc86de-7aef-4122-85da-ef448195c025 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:42.633133 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc86de-7aef-4122-85da-ef448195c025 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:42.634674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1e31a-4a3b-4a16-9d91-7b39d45ae167 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:42.668658 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326bdc1-f560-4eac-8d90-872cf9e562c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:42.669685 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326bdc1-f560-4eac-8d90-872cf9e562c1 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:42.670227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326bdc1-f560-4eac-8d90-872cf9e562c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:42.670725 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326bdc1-f560-4eac-8d90-872cf9e562c1 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:42.671763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326bdc1-f560-4eac-8d90-872cf9e562c1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:42.672268 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326bdc1-f560-4eac-8d90-872cf9e562c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:42.673273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5457e95-999a-4998-9ad4-b3fc66ca0477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:42.673930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5457e95-999a-4998-9ad4-b3fc66ca0477 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:42.674247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5457e95-999a-4998-9ad4-b3fc66ca0477 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:42.674528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5457e95-999a-4998-9ad4-b3fc66ca0477 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:42.674825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326bdc1-f560-4eac-8d90-872cf9e562c1 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:42.675372 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5457e95-999a-4998-9ad4-b3fc66ca0477 None admin] RBAC: Adding query filter params (role.id=5c87e804d0764309ba08ad6e06b23a87, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=b4d10b258a53484da22950e2b1db2e41) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:42.675688 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5457e95-999a-4998-9ad4-b3fc66ca0477 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:42.677107 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5457e95-999a-4998-9ad4-b3fc66ca0477 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:42.684768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 431/869] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:58:42 2022] GET /identity/v3/role_assignments?user.id=b4d10b258a53484da22950e2b1db2e41&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:42.701742 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 439/870] 10.222.0.65 () {56 vars in 1149 bytes} [Tue May 10 19:58:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:42.707691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5457e95-999a-4998-9ad4-b3fc66ca0477 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:42.816081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5457e95-999a-4998-9ad4-b3fc66ca0477 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:42.853954 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce387ec-c9c0-4246-bdf1-845894896c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:42.855008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce387ec-c9c0-4246-bdf1-845894896c4c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:42.855500 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce387ec-c9c0-4246-bdf1-845894896c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:42.856013 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce387ec-c9c0-4246-bdf1-845894896c4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:42.926356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce387ec-c9c0-4246-bdf1-845894896c4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:42.927733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce387ec-c9c0-4246-bdf1-845894896c4c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:43.026634 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 432/871] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:58:42 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 19:58:43.031832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326bdc1-f560-4eac-8d90-872cf9e562c1 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:43.133440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326bdc1-f560-4eac-8d90-872cf9e562c1 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:43.168395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21c6c2-08d2-4841-9e9b-f04de281882f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:43.169195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21c6c2-08d2-4841-9e9b-f04de281882f None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:43.169480 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21c6c2-08d2-4841-9e9b-f04de281882f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:43.169757 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21c6c2-08d2-4841-9e9b-f04de281882f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:43.180723 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21c6c2-08d2-4841-9e9b-f04de281882f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:43.182261 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21c6c2-08d2-4841-9e9b-f04de281882f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:43.202203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-1c21c6c2-08d2-4841-9e9b-f04de281882f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:43.202203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:43.202203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:43.202203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:43.202203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:43.202203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:43.202203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:43.202203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:43.202203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:43.202203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:43.202203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:43.202203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:43.202203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:43.202203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:43.202203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:43.203167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:43.203167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:43.203167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:43.203167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:43.203167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:43.203167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:43.203167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:43.203167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:43.203167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:43.203167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:43.203167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:43.203167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:43.203167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:43.203167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:43.203167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:43.203167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:43.203167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:43.204689 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:43.204689 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:43.204689 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:43.204689 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:43.204689 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 440/872] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:43.209767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce387ec-c9c0-4246-bdf1-845894896c4c None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:43.323555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce387ec-c9c0-4246-bdf1-845894896c4c None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:43.357290 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02593cc8-eaef-4477-a504-17ebc345b7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:43.357955 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02593cc8-eaef-4477-a504-17ebc345b7fd None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:43.358278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02593cc8-eaef-4477-a504-17ebc345b7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:43.358562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02593cc8-eaef-4477-a504-17ebc345b7fd None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:43.359292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02593cc8-eaef-4477-a504-17ebc345b7fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:43.359601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02593cc8-eaef-4477-a504-17ebc345b7fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:43.361499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02593cc8-eaef-4477-a504-17ebc345b7fd None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:43.373418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 433/873] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:43.379332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21c6c2-08d2-4841-9e9b-f04de281882f None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:43.483166 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21c6c2-08d2-4841-9e9b-f04de281882f None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:43.521256 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ca39d-306a-4c3c-a706-00ee7de8c659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:43.522046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ca39d-306a-4c3c-a706-00ee7de8c659 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:43.522352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ca39d-306a-4c3c-a706-00ee7de8c659 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:43.522626 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ca39d-306a-4c3c-a706-00ee7de8c659 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:43.529654 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ca39d-306a-4c3c-a706-00ee7de8c659 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:43.531118 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ca39d-306a-4c3c-a706-00ee7de8c659 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:43.545070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-040ca39d-306a-4c3c-a706-00ee7de8c659 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:43.545070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:43.545070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:43.545070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:43.545070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:43.545070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:43.545070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:43.545070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:43.545070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:43.545070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:43.545070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:43.545070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:43.545070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:43.545070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:43.545070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:43.547160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:43.547160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:43.547160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:43.547160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:43.547160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:43.547160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:43.547160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:43.547160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:43.547160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:43.547160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:43.547160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:43.547160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:43.547160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:43.547160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:43.547160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:43.547160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:43.547160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:43.549833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:43.549833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:43.549833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:43.549833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:43.549833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:43.549833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:43.549833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:43.549833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:43.549833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 441/874] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:43.554203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02593cc8-eaef-4477-a504-17ebc345b7fd None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:43.653797 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02593cc8-eaef-4477-a504-17ebc345b7fd None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:43.662882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036954d8-65de-44f3-8372-ed16afc6b4d9 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:43.663493 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036954d8-65de-44f3-8372-ed16afc6b4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:43.663988 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036954d8-65de-44f3-8372-ed16afc6b4d9 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:43.665815 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 442/875] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:58:43.670611 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657e4cf-d4bb-4a3c-8db6-66216da3cb0f None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:43.670898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657e4cf-d4bb-4a3c-8db6-66216da3cb0f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:43.671186 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657e4cf-d4bb-4a3c-8db6-66216da3cb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:43.690280 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-f657e4cf-d4bb-4a3c-8db6-66216da3cb0f None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:43.691342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e21024-8f0c-4071-92cb-3fdeed34c8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:43.691582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f657e4cf-d4bb-4a3c-8db6-66216da3cb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:43.692440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e21024-8f0c-4071-92cb-3fdeed34c8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:43.692932 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e21024-8f0c-4071-92cb-3fdeed34c8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:43.693568 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e21024-8f0c-4071-92cb-3fdeed34c8b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:43.695368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e21024-8f0c-4071-92cb-3fdeed34c8b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:43.695898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e21024-8f0c-4071-92cb-3fdeed34c8b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:43.697597 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e21024-8f0c-4071-92cb-3fdeed34c8b4 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:43.711319 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 434/876] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:43.718667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e21024-8f0c-4071-92cb-3fdeed34c8b4 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:43.767610 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 443/877] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:58:43.793552 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657e4cf-d4bb-4a3c-8db6-66216da3cb0f None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:43.827494 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e21024-8f0c-4071-92cb-3fdeed34c8b4 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:43.862533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be823f-70f7-49f6-9331-cc48d6655bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:43.863392 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be823f-70f7-49f6-9331-cc48d6655bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:43.863670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be823f-70f7-49f6-9331-cc48d6655bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:43.863946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be823f-70f7-49f6-9331-cc48d6655bb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:43.864787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be823f-70f7-49f6-9331-cc48d6655bb9 None admin] RBAC: Adding query filter params (role.id=55528a93cfa64162b2f9574fb5cdde79, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=7ec2605fd1b84e0ea134b943ed33a605) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:43.865098 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be823f-70f7-49f6-9331-cc48d6655bb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:43.867275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be823f-70f7-49f6-9331-cc48d6655bb9 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:43.880105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 435/878] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:58:43 2022] GET /identity/v3/role_assignments?user.id=7ec2605fd1b84e0ea134b943ed33a605&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=55528a93cfa64162b2f9574fb5cdde79 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:43.905486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657e4cf-d4bb-4a3c-8db6-66216da3cb0f None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:43.942765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6995b9-4d35-4c84-a953-4394d3607be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:43.943586 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6995b9-4d35-4c84-a953-4394d3607be8 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:43.943871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6995b9-4d35-4c84-a953-4394d3607be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:43.944151 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6995b9-4d35-4c84-a953-4394d3607be8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:43.944823 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6995b9-4d35-4c84-a953-4394d3607be8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:43.946277 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6995b9-4d35-4c84-a953-4394d3607be8 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:43.959721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-0c6995b9-4d35-4c84-a953-4394d3607be8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 19:58:43.959721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:43.959721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:43.959721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:43.959721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:43.959721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:43.959721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:43.959721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:43.959721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 19:58:43.959721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:43.959721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:43.959721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:43.959721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:43.959721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:43.959721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:43.959721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:43.960737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:43.960737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:43.960737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:43.960737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:43.960737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:43.960737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:43.960737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:43.960737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:43.960737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:43.960737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:43.960737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:43.960737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:43.960737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:43.960737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:43.960737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:43.960737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:43.961576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:43.961576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:43.961576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:43.961576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:43.961576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:43.961576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:43.961576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:43.961576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:43.961576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:43.961576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:43.961576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:43.961576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:43.961576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:43.961576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 19:58:43.961576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:43.961576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 444/879] 10.222.0.65 () {56 vars in 1132 bytes} [Tue May 10 19:58:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:43.967405 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be823f-70f7-49f6-9331-cc48d6655bb9 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:44.076712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be823f-70f7-49f6-9331-cc48d6655bb9 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:44.110015 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928242a5-6f56-4dfc-8ed4-d87af4722237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:44.110712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928242a5-6f56-4dfc-8ed4-d87af4722237 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:44.110995 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928242a5-6f56-4dfc-8ed4-d87af4722237 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:44.111269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928242a5-6f56-4dfc-8ed4-d87af4722237 None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:44.111984 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928242a5-6f56-4dfc-8ed4-d87af4722237 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:44.112288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928242a5-6f56-4dfc-8ed4-d87af4722237 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:44.113910 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928242a5-6f56-4dfc-8ed4-d87af4722237 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:44.118733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 436/880] 10.222.0.65 () {56 vars in 1126 bytes} [Tue May 10 19:58:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:44.124315 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6995b9-4d35-4c84-a953-4394d3607be8 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:44.231554 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6995b9-4d35-4c84-a953-4394d3607be8 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:44.266982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a8def-d579-4253-bc98-f69965ac0896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:44.268373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a8def-d579-4253-bc98-f69965ac0896 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:44.269365 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a8def-d579-4253-bc98-f69965ac0896 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:44.270007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a8def-d579-4253-bc98-f69965ac0896 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:44.271364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a8def-d579-4253-bc98-f69965ac0896 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:44.271823 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a8def-d579-4253-bc98-f69965ac0896 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:44.273445 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a8def-d579-4253-bc98-f69965ac0896 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:44.279339 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 445/881] 10.222.0.65 () {56 vars in 1126 bytes} [Tue May 10 19:58:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:44.285462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928242a5-6f56-4dfc-8ed4-d87af4722237 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:44.381589 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965a449-de83-4b4a-8816-9e2bd40784ea None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:44.382317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965a449-de83-4b4a-8816-9e2bd40784ea None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:44.382317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965a449-de83-4b4a-8816-9e2bd40784ea None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:44.384998 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 446/882] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:44.389221 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92f585-58d2-4a30-bc5f-37b8fa1f0684 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:44.389612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92f585-58d2-4a30-bc5f-37b8fa1f0684 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:44.390013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92f585-58d2-4a30-bc5f-37b8fa1f0684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:44.394480 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928242a5-6f56-4dfc-8ed4-d87af4722237 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:44.407712 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-8a92f585-58d2-4a30-bc5f-37b8fa1f0684 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:44.408961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8a92f585-58d2-4a30-bc5f-37b8fa1f0684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:44.443447 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55711cec-c266-4998-a434-5398fc356aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:44.444155 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55711cec-c266-4998-a434-5398fc356aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:44.444424 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55711cec-c266-4998-a434-5398fc356aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:44.444710 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55711cec-c266-4998-a434-5398fc356aa9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:44.445541 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55711cec-c266-4998-a434-5398fc356aa9 None admin] RBAC: Adding query filter params (interface=public, service_id=87456968545a461190c0e90391957959, region_id=RegionOne) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:44.445855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55711cec-c266-4998-a434-5398fc356aa9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:44.447352 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55711cec-c266-4998-a434-5398fc356aa9 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:44.453267 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 437/883] 10.222.0.65 () {56 vars in 1251 bytes} [Tue May 10 19:58:44 2022] GET /identity/v3/endpoints?service_id=87456968545a461190c0e90391957959&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:44.461745 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55711cec-c266-4998-a434-5398fc356aa9 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:44.493763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 447/884] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:58:44.515765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92f585-58d2-4a30-bc5f-37b8fa1f0684 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:44.582388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55711cec-c266-4998-a434-5398fc356aa9 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:44.617943 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1621c7-f279-4374-9068-392eacfb4f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:44.618659 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1621c7-f279-4374-9068-392eacfb4f78 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:44.618949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1621c7-f279-4374-9068-392eacfb4f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:44.619227 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1621c7-f279-4374-9068-392eacfb4f78 None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:44.619922 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1621c7-f279-4374-9068-392eacfb4f78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:44.621288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1621c7-f279-4374-9068-392eacfb4f78 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:44.634621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 438/885] 10.222.0.65 () {56 vars in 1087 bytes} [Tue May 10 19:58:44 2022] GET /identity/v3/services => generated 1268 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 19:58:44.636436 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92f585-58d2-4a30-bc5f-37b8fa1f0684 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:44.676054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c820c5a-bf51-4c81-af13-16f298d1a058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:44.676864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c820c5a-bf51-4c81-af13-16f298d1a058 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:44.677150 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c820c5a-bf51-4c81-af13-16f298d1a058 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:44.677424 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c820c5a-bf51-4c81-af13-16f298d1a058 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:44.678224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c820c5a-bf51-4c81-af13-16f298d1a058 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:44.679618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c820c5a-bf51-4c81-af13-16f298d1a058 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:44.692649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-6c820c5a-bf51-4c81-af13-16f298d1a058 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 19:58:44.692649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:44.692649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:44.692649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:44.692649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:44.692649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:44.692649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:44.692649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:44.692649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 19:58:44.692649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:44.692649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:44.692649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:44.692649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:44.692649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:44.692649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:44.692649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:44.692649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:44.693781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:44.693781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:44.693781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:44.693781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:44.693781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:44.693781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:44.693781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:44.693781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:44.693781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:44.693781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:44.693781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:44.693781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:44.693781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:44.693781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:44.693781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:44.693781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:44.694714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:44.694714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:44.694714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:44.694714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:44.694714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:44.694714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:44.694714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:44.694714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:44.694714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:44.694714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:44.694714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:44.694714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:44.694714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 19:58:44.694714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:44.694714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 448/886] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:58:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:44.700121 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1621c7-f279-4374-9068-392eacfb4f78 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:44.818408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1621c7-f279-4374-9068-392eacfb4f78 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:44.857155 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9981c53-a8d8-40e7-9db8-254cb5ec72ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:44.857883 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9981c53-a8d8-40e7-9db8-254cb5ec72ab None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:44.858194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9981c53-a8d8-40e7-9db8-254cb5ec72ab None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:44.858520 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9981c53-a8d8-40e7-9db8-254cb5ec72ab None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:44.859236 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9981c53-a8d8-40e7-9db8-254cb5ec72ab None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:44.859543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9981c53-a8d8-40e7-9db8-254cb5ec72ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:44.860944 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9981c53-a8d8-40e7-9db8-254cb5ec72ab None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:44.866385 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 439/887] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:58:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:44.871951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c820c5a-bf51-4c81-af13-16f298d1a058 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:44.973211 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c820c5a-bf51-4c81-af13-16f298d1a058 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:45.007297 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84049d20-270f-4f73-87da-908993082ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:45.008306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84049d20-270f-4f73-87da-908993082ead None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:45.008819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84049d20-270f-4f73-87da-908993082ead None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:45.009310 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84049d20-270f-4f73-87da-908993082ead None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:45.010253 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84049d20-270f-4f73-87da-908993082ead None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:45.010848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84049d20-270f-4f73-87da-908993082ead None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:45.012395 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53db39-69c9-4aee-a4f6-21bc9e8ed82e None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:45.012655 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84049d20-270f-4f73-87da-908993082ead None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:45.012770 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53db39-69c9-4aee-a4f6-21bc9e8ed82e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:45.012954 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53db39-69c9-4aee-a4f6-21bc9e8ed82e None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:45.014582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 440/888] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:45.019674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 449/889] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:58:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:45.020386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6f8b0-4e02-499d-8369-d074e2885bdb None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:45.020682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6f8b0-4e02-499d-8369-d074e2885bdb None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:45.020965 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6f8b0-4e02-499d-8369-d074e2885bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:45.025200 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84049d20-270f-4f73-87da-908993082ead None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:45.039567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-f0f6f8b0-4e02-499d-8369-d074e2885bdb None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:45.040755 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-f0f6f8b0-4e02-499d-8369-d074e2885bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:45.123200 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 441/890] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:58:45.135509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84049d20-270f-4f73-87da-908993082ead None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:45.145620 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6f8b0-4e02-499d-8369-d074e2885bdb None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:45.173474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ca53e-cd38-4e69-bc06-eee4fb8c4d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:45.174515 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ca53e-cd38-4e69-bc06-eee4fb8c4d8e None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:45.175013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ca53e-cd38-4e69-bc06-eee4fb8c4d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:45.175519 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ca53e-cd38-4e69-bc06-eee4fb8c4d8e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:45.176534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ca53e-cd38-4e69-bc06-eee4fb8c4d8e None admin] RBAC: Adding query filter params (interface=public, service_id=3536c54362c6411e9ef8ce7eeacdd5d7, region_id=RegionOne) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:45.177075 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ca53e-cd38-4e69-bc06-eee4fb8c4d8e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:45.178679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ca53e-cd38-4e69-bc06-eee4fb8c4d8e None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:45.191903 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 450/891] 10.222.0.65 () {56 vars in 1251 bytes} [Tue May 10 19:58:45 2022] GET /identity/v3/endpoints?service_id=3536c54362c6411e9ef8ce7eeacdd5d7&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:45.193310 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ca53e-cd38-4e69-bc06-eee4fb8c4d8e None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:45.255871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6f8b0-4e02-499d-8369-d074e2885bdb None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:45.302891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510998e-1ed0-4a8f-acea-365d892f9467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:45.303742 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510998e-1ed0-4a8f-acea-365d892f9467 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:45.305091 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510998e-1ed0-4a8f-acea-365d892f9467 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:45.305091 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510998e-1ed0-4a8f-acea-365d892f9467 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:45.305091 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510998e-1ed0-4a8f-acea-365d892f9467 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:45.308235 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510998e-1ed0-4a8f-acea-365d892f9467 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:45.321347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-e510998e-1ed0-4a8f-acea-365d892f9467 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 19:58:45.321347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:45.321347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:45.321347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:45.321347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:45.321347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:45.321347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:45.321347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:45.321347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 19:58:45.321347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:45.321347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:45.321347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:45.321347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:45.321347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:45.321347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:45.321347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:45.321347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:45.322537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:45.322537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:45.322537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:45.322537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:45.322537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:45.322537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:45.322537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:45.322537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:45.322537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:45.322537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:45.322537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:45.322537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:45.322537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:45.322537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:45.322537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:45.322537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:45.329497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:45.329497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:45.329497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:45.329497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:45.329497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:45.329497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:45.329497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:45.329497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:45.329497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:45.329497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:45.329497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:45.329497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:45.329497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 19:58:45.329497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:45.329497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 442/892] 10.222.0.65 () {56 vars in 1126 bytes} [Tue May 10 19:58:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:45.330823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510998e-1ed0-4a8f-acea-365d892f9467 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:45.343754 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ca53e-cd38-4e69-bc06-eee4fb8c4d8e None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:45.383638 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59152eae-9c9d-4434-90c2-a47962977a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:45.385193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59152eae-9c9d-4434-90c2-a47962977a40 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:45.385716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59152eae-9c9d-4434-90c2-a47962977a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:45.386226 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59152eae-9c9d-4434-90c2-a47962977a40 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:45.387137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59152eae-9c9d-4434-90c2-a47962977a40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:45.388832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59152eae-9c9d-4434-90c2-a47962977a40 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:45.402533 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 451/893] 10.222.0.65 () {56 vars in 1087 bytes} [Tue May 10 19:58:45 2022] GET /identity/v3/services => generated 1268 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 19:58:45.436894 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510998e-1ed0-4a8f-acea-365d892f9467 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:45.475040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3deee-ced4-4816-a3bb-0eceb43939db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:45.475710 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3deee-ced4-4816-a3bb-0eceb43939db None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:45.475994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3deee-ced4-4816-a3bb-0eceb43939db None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:45.476273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3deee-ced4-4816-a3bb-0eceb43939db None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:45.477000 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3deee-ced4-4816-a3bb-0eceb43939db None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:45.477308 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3deee-ced4-4816-a3bb-0eceb43939db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:45.478723 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3deee-ced4-4816-a3bb-0eceb43939db None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:45.484439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 443/894] 10.222.0.65 () {56 vars in 1122 bytes} [Tue May 10 19:58:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:45.491039 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59152eae-9c9d-4434-90c2-a47962977a40 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:45.604308 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59152eae-9c9d-4434-90c2-a47962977a40 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:45.642598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a397e-b662-4271-9ac3-d29a45bed79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:45.643661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a397e-b662-4271-9ac3-d29a45bed79f None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:45.644150 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a397e-b662-4271-9ac3-d29a45bed79f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:45.644637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a397e-b662-4271-9ac3-d29a45bed79f None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:45.645567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a397e-b662-4271-9ac3-d29a45bed79f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:45.646075 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a397e-b662-4271-9ac3-d29a45bed79f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:45.647687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a397e-b662-4271-9ac3-d29a45bed79f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:45.652798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 452/895] 10.222.0.65 () {56 vars in 1122 bytes} [Tue May 10 19:58:45 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:46.330494 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a531c-8666-44a9-bf89-b184e5fd05a2 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:46.330830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a531c-8666-44a9-bf89-b184e5fd05a2 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:46.331051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a531c-8666-44a9-bf89-b184e5fd05a2 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:46.332691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 444/896] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:46.338001 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfb94e-f989-43b2-a5f4-3e70b456afde None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:46.338321 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfb94e-f989-43b2-a5f4-3e70b456afde None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:46.338598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfb94e-f989-43b2-a5f4-3e70b456afde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:46.356417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-05cfb94e-f989-43b2-a5f4-3e70b456afde None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:46.357583 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-05cfb94e-f989-43b2-a5f4-3e70b456afde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:46.431994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 453/897] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:58:46.455377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a531c-8666-44a9-bf89-b184e5fd05a2 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:46.560532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a531c-8666-44a9-bf89-b184e5fd05a2 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:46.596986 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b04b67-b083-439e-9225-f39457a880ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:46.597712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b04b67-b083-439e-9225-f39457a880ed None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:46.597994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b04b67-b083-439e-9225-f39457a880ed None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:46.598297 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b04b67-b083-439e-9225-f39457a880ed None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:46.609015 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b04b67-b083-439e-9225-f39457a880ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:46.617442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-75b04b67-b083-439e-9225-f39457a880ed None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 19:58:46.617442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:46.617442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:46.617442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:46.617442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:46.617442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:46.617442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:46.617442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:46.617442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:46.617442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:46.617442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:46.617442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:46.617442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:46.617442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:46.617442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:46.617442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:46.619207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:46.619207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:46.619207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:46.619207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:46.619207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:46.619207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:46.619207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:46.619207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:46.619207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:46.619207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:46.619207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:46.619207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:46.619207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:46.619207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:46.619207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:46.619207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:46.621177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:46.621177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:46.621177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:46.621177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:46.621177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:46.621177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:46.621177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:46.621177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:46.621177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:46.621177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:46.621177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 19:58:46.621177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:46.621177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 445/898] 10.222.0.65 () {56 vars in 1120 bytes} [Tue May 10 19:58:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:46.624723 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cfb94e-f989-43b2-a5f4-3e70b456afde None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:46.724224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cfb94e-f989-43b2-a5f4-3e70b456afde None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:46.758598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c99223-08ab-491e-8c30-fc9423153bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:46.759262 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c99223-08ab-491e-8c30-fc9423153bce None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:46.759544 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c99223-08ab-491e-8c30-fc9423153bce None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:46.759818 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c99223-08ab-491e-8c30-fc9423153bce None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:46.760564 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c99223-08ab-491e-8c30-fc9423153bce None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:46.760870 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c99223-08ab-491e-8c30-fc9423153bce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:46.762373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c99223-08ab-491e-8c30-fc9423153bce None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:46.771709 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 454/899] 10.222.0.65 () {56 vars in 1115 bytes} [Tue May 10 19:58:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:46.779256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b04b67-b083-439e-9225-f39457a880ed None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:46.885862 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b04b67-b083-439e-9225-f39457a880ed None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:46.920215 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac4365-72b2-421d-a1e5-9bca957578e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:46.920997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac4365-72b2-421d-a1e5-9bca957578e8 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:46.921278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac4365-72b2-421d-a1e5-9bca957578e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:46.921554 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac4365-72b2-421d-a1e5-9bca957578e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:46.931614 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac4365-72b2-421d-a1e5-9bca957578e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:46.933058 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac4365-72b2-421d-a1e5-9bca957578e8 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:46.950971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-cbac4365-72b2-421d-a1e5-9bca957578e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:46.950971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:46.950971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:46.950971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:46.950971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:46.950971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:46.950971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:46.950971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:46.950971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:46.950971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:46.950971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:46.950971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:46.950971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:46.950971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:46.950971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:46.953081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:46.953081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:46.953081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:46.953081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:46.953081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:46.953081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:46.953081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:46.953081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:46.953081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:46.953081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:46.953081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:46.953081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:46.953081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:46.953081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:46.953081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:46.953081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:46.953081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:46.955883 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:46.955883 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:46.955883 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:46.955883 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:46.955883 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 446/900] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:46.962107 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c99223-08ab-491e-8c30-fc9423153bce None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:46.994223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e198e3b-4bd4-4913-acd4-1c440c1f6639 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:46.994541 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e198e3b-4bd4-4913-acd4-1c440c1f6639 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:46.994785 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e198e3b-4bd4-4913-acd4-1c440c1f6639 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:46.996387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 447/901] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:58:47.002006 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f5744-f52d-47b6-a64e-9b5795c68fb3 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:47.002334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f5744-f52d-47b6-a64e-9b5795c68fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:47.002615 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f5744-f52d-47b6-a64e-9b5795c68fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:47.019153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-4b3f5744-f52d-47b6-a64e-9b5795c68fb3 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:47.020325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4b3f5744-f52d-47b6-a64e-9b5795c68fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:47.060449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c99223-08ab-491e-8c30-fc9423153bce None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:47.094466 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4942293-f49e-4c8c-bf01-c6d19a77f803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:47.095124 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4942293-f49e-4c8c-bf01-c6d19a77f803 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:47.095403 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4942293-f49e-4c8c-bf01-c6d19a77f803 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:47.095679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4942293-f49e-4c8c-bf01-c6d19a77f803 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:47.096403 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4942293-f49e-4c8c-bf01-c6d19a77f803 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:47.096710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4942293-f49e-4c8c-bf01-c6d19a77f803 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:47.098148 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 448/902] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:58:47.098336 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4942293-f49e-4c8c-bf01-c6d19a77f803 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:47.111354 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f5744-f52d-47b6-a64e-9b5795c68fb3 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:47.112355 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 455/903] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:47.117843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4942293-f49e-4c8c-bf01-c6d19a77f803 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:47.211159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f5744-f52d-47b6-a64e-9b5795c68fb3 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:47.217155 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4942293-f49e-4c8c-bf01-c6d19a77f803 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:47.245963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb8232-12cd-4274-9df3-8aacb2789085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:47.246804 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb8232-12cd-4274-9df3-8aacb2789085 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:47.247082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb8232-12cd-4274-9df3-8aacb2789085 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:47.247353 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb8232-12cd-4274-9df3-8aacb2789085 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:47.248019 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb8232-12cd-4274-9df3-8aacb2789085 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:47.249533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb8232-12cd-4274-9df3-8aacb2789085 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:47.252306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d698f-6c8d-4105-b211-d361faebfc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:47.253117 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d698f-6c8d-4105-b211-d361faebfc31 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:47.253720 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d698f-6c8d-4105-b211-d361faebfc31 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:47.254001 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d698f-6c8d-4105-b211-d361faebfc31 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:47.261437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d698f-6c8d-4105-b211-d361faebfc31 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:47.262223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-1bbb8232-12cd-4274-9df3-8aacb2789085 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 19:58:47.262223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:47.262223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:47.262223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:47.262223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:47.262223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:47.262223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:47.262223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:47.262223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 19:58:47.262223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:47.262223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:47.262223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:47.262223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:47.262223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:47.262223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:47.262223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:47.264512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:47.264512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:47.264512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:47.264512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:47.264512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:47.264512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:47.264512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:47.264512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:47.264512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:47.264512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:47.264512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:47.264512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:47.264512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:47.264512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:47.264512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:47.264512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:47.266607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:47.266607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:47.266607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:47.266607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:47.266607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:47.266607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:47.266607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:47.266607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:47.266607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:47.266607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:47.266607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:47.266607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:47.266607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:47.266607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 19:58:47.266607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:47.268579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d698f-6c8d-4105-b211-d361faebfc31 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:47.268744 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 449/904] 10.222.0.65 () {56 vars in 1132 bytes} [Tue May 10 19:58:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:47.270190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb8232-12cd-4274-9df3-8aacb2789085 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:47.279067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-890d698f-6c8d-4105-b211-d361faebfc31 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 19:58:47.279067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:47.279067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:47.279067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:47.279067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:47.279067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:47.279067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:47.279067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:47.279067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:47.279067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:47.279067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:47.279067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:47.279067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:47.279067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:47.279067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:47.279067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:47.280271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:47.280271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:47.280271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:47.280271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:47.280271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:47.280271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:47.280271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:47.280271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:47.280271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:47.280271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:47.280271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:47.280271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:47.280271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:47.280271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:47.280271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:47.280271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:47.280271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:47.280271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:47.281273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:47.281273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:47.281273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:47.281273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:47.281273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:47.281273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:47.281273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:47.281273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:47.281273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 19:58:47.281273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:47.285835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 456/905] 10.222.0.65 () {56 vars in 1117 bytes} [Tue May 10 19:58:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:47.286789 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d698f-6c8d-4105-b211-d361faebfc31 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:47.382719 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb8232-12cd-4274-9df3-8aacb2789085 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:47.390453 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d698f-6c8d-4105-b211-d361faebfc31 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:47.417606 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e0c49-9985-47c2-b8ad-bd10fa71886d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:47.418647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e0c49-9985-47c2-b8ad-bd10fa71886d None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:47.419137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e0c49-9985-47c2-b8ad-bd10fa71886d None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:47.419600 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e0c49-9985-47c2-b8ad-bd10fa71886d None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:47.420513 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e0c49-9985-47c2-b8ad-bd10fa71886d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:47.420999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e0c49-9985-47c2-b8ad-bd10fa71886d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:47.423327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e0c49-9985-47c2-b8ad-bd10fa71886d None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:47.429687 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 450/906] 10.222.0.65 () {56 vars in 1126 bytes} [Tue May 10 19:58:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:47.432138 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b1da9-dc78-4309-a746-31afc1904251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:47.433328 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b1da9-dc78-4309-a746-31afc1904251 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:47.433791 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b1da9-dc78-4309-a746-31afc1904251 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:47.434259 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b1da9-dc78-4309-a746-31afc1904251 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:47.435277 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e0c49-9985-47c2-b8ad-bd10fa71886d None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:47.438876 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b1da9-dc78-4309-a746-31afc1904251 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:47.439935 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b1da9-dc78-4309-a746-31afc1904251 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:47.440608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b1da9-dc78-4309-a746-31afc1904251 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:47.463851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 457/907] 10.222.0.65 () {56 vars in 1149 bytes} [Tue May 10 19:58:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:47.464863 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b1da9-dc78-4309-a746-31afc1904251 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:47.548799 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e0c49-9985-47c2-b8ad-bd10fa71886d None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:47.584251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9968d-e3a6-431b-97c9-06de2f7ed1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:47.584681 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9968d-e3a6-431b-97c9-06de2f7ed1ee None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:47.584966 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9968d-e3a6-431b-97c9-06de2f7ed1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:47.585243 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9968d-e3a6-431b-97c9-06de2f7ed1ee None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:47.585961 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9968d-e3a6-431b-97c9-06de2f7ed1ee None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:47.586307 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9968d-e3a6-431b-97c9-06de2f7ed1ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:47.587714 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9968d-e3a6-431b-97c9-06de2f7ed1ee None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:47.597690 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 451/908] 10.222.0.65 () {56 vars in 1126 bytes} [Tue May 10 19:58:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:47.598644 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9968d-e3a6-431b-97c9-06de2f7ed1ee None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:47.600371 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b1da9-dc78-4309-a746-31afc1904251 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:47.635860 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04598e08-9f22-44dc-89a3-ea8259309724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:47.636518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04598e08-9f22-44dc-89a3-ea8259309724 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:47.636998 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04598e08-9f22-44dc-89a3-ea8259309724 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:47.637105 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04598e08-9f22-44dc-89a3-ea8259309724 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:47.705669 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04598e08-9f22-44dc-89a3-ea8259309724 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:47.707349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04598e08-9f22-44dc-89a3-ea8259309724 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:47.721895 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9968d-e3a6-431b-97c9-06de2f7ed1ee None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:47.753690 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad1745-316e-4f00-a2de-d69e9e61a26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:47.754350 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad1745-316e-4f00-a2de-d69e9e61a26e None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:47.754636 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad1745-316e-4f00-a2de-d69e9e61a26e None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:47.754912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad1745-316e-4f00-a2de-d69e9e61a26e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:47.755657 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad1745-316e-4f00-a2de-d69e9e61a26e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:47.757019 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad1745-316e-4f00-a2de-d69e9e61a26e None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:47.807862 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 458/909] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:58:47 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 19:58:47.809213 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bcc5c-5aba-41e9-af59-b5a7c49cebdb None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:47.809498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bcc5c-5aba-41e9-af59-b5a7c49cebdb None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:47.809775 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bcc5c-5aba-41e9-af59-b5a7c49cebdb None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:47.811240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 459/910] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:47.815081 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bcc5c-5aba-41e9-af59-b5a7c49cebdb None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:47.874116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 452/911] 10.222.0.65 () {60 vars in 1144 bytes} [Tue May 10 19:58:47 2022] POST /identity/v3/endpoints => generated 347 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:58:47.874116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c59dd-3796-400b-aba7-087180f5755e None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:47.874116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c59dd-3796-400b-aba7-087180f5755e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:47.874116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c59dd-3796-400b-aba7-087180f5755e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:47.888342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-7a1c59dd-3796-400b-aba7-087180f5755e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:47.889787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7a1c59dd-3796-400b-aba7-087180f5755e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:47.942811 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bcc5c-5aba-41e9-af59-b5a7c49cebdb None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:47.979799 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 453/912] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:47.981335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11fd7b-c6a1-4cad-b6f7-151bd51cbc86 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:47.981658 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11fd7b-c6a1-4cad-b6f7-151bd51cbc86 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:47.981915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11fd7b-c6a1-4cad-b6f7-151bd51cbc86 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:47.987602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 454/913] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:47.989287 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d071f-aaab-41cb-b7d3-96b97879cfb9 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:47.989588 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d071f-aaab-41cb-b7d3-96b97879cfb9 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:47.989788 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dec7a5-dea4-4d00-856c-0003100a00df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:47.989894 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d071f-aaab-41cb-b7d3-96b97879cfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:47.990962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dec7a5-dea4-4d00-856c-0003100a00df None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:47.991479 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dec7a5-dea4-4d00-856c-0003100a00df None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:47.991944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dec7a5-dea4-4d00-856c-0003100a00df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:48.004366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dec7a5-dea4-4d00-856c-0003100a00df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:48.005874 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dec7a5-dea4-4d00-856c-0003100a00df None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:48.009814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-cb1d071f-aaab-41cb-b7d3-96b97879cfb9 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:48.011338 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-cb1d071f-aaab-41cb-b7d3-96b97879cfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:48.025178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-a4dec7a5-dea4-4d00-856c-0003100a00df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:48.025178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:48.025178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:48.025178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:48.025178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:48.025178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:48.025178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:48.025178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:48.025178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:48.025178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:48.025178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:48.025178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:48.025178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:48.025178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:48.025178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:48.026183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:48.026183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:48.026183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:48.026183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:48.026183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:48.026183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:48.026183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:48.026183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:48.026183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:48.026183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:48.026183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:48.026183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:48.026183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:48.026183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:48.026183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:48.026183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:48.026183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:48.027503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:48.027503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:48.027503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:48.027503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:48.027503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 460/914] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:48.027783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dec7a5-dea4-4d00-856c-0003100a00df None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:48.088603 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 455/915] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:48.088603 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d071f-aaab-41cb-b7d3-96b97879cfb9 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:48.139228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dec7a5-dea4-4d00-856c-0003100a00df None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:48.172596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8cbd2-fb09-4c4a-a9d3-679d5ac2ad05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:48.173383 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8cbd2-fb09-4c4a-a9d3-679d5ac2ad05 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:48.173661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8cbd2-fb09-4c4a-a9d3-679d5ac2ad05 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:48.173926 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8cbd2-fb09-4c4a-a9d3-679d5ac2ad05 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:48.174609 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8cbd2-fb09-4c4a-a9d3-679d5ac2ad05 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:48.175962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8cbd2-fb09-4c4a-a9d3-679d5ac2ad05 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:48.186735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d071f-aaab-41cb-b7d3-96b97879cfb9 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:48.188724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-d6c8cbd2-fb09-4c4a-a9d3-679d5ac2ad05 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 19:58:48.188724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:48.188724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:48.188724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:48.188724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:48.188724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:48.188724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:48.188724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:48.188724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 19:58:48.188724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:48.188724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:48.188724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:48.188724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:48.188724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:48.188724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:48.188724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:48.188724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:48.189672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:48.189672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:48.189672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:48.189672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:48.189672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:48.189672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:48.189672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:48.189672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:48.189672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:48.189672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:48.189672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:48.189672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:48.189672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:48.189672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:48.189672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:48.189672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:48.190539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:48.190539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:48.190539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:48.190539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:48.190539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:48.190539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:48.190539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:48.190539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:48.190539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:48.190539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:48.190539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:48.190539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:48.190539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 19:58:48.190539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:48.190539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 461/916] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:58:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:48.191937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8cbd2-fb09-4c4a-a9d3-679d5ac2ad05 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:48.226356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1550c-28f9-4508-a415-7d2496020e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:48.227027 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1550c-28f9-4508-a415-7d2496020e16 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:48.227313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1550c-28f9-4508-a415-7d2496020e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:48.227579 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1550c-28f9-4508-a415-7d2496020e16 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:48.228304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e1550c-28f9-4508-a415-7d2496020e16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:48.228604 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e1550c-28f9-4508-a415-7d2496020e16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:48.230009 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e1550c-28f9-4508-a415-7d2496020e16 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:48.245562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 456/917] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:48.245562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1550c-28f9-4508-a415-7d2496020e16 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:48.298677 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8cbd2-fb09-4c4a-a9d3-679d5ac2ad05 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:48.332964 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc98101-ffb2-4b3a-96e2-50497061f4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:48.333628 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc98101-ffb2-4b3a-96e2-50497061f4c2 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:48.333914 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc98101-ffb2-4b3a-96e2-50497061f4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:48.334217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc98101-ffb2-4b3a-96e2-50497061f4c2 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:48.334942 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc98101-ffb2-4b3a-96e2-50497061f4c2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:48.336308 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc98101-ffb2-4b3a-96e2-50497061f4c2 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:48.339213 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1550c-28f9-4508-a415-7d2496020e16 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:48.371203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe9b29-8516-478c-89b3-6aa4fcb4ed7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:48.371857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe9b29-8516-478c-89b3-6aa4fcb4ed7b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:48.372136 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe9b29-8516-478c-89b3-6aa4fcb4ed7b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:48.372410 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe9b29-8516-478c-89b3-6aa4fcb4ed7b None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:48.373114 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe9b29-8516-478c-89b3-6aa4fcb4ed7b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:48.373419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe9b29-8516-478c-89b3-6aa4fcb4ed7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:48.374810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe9b29-8516-478c-89b3-6aa4fcb4ed7b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:48.379746 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 457/918] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:58:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:48.380498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe9b29-8516-478c-89b3-6aa4fcb4ed7b None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:48.404146 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 462/919] 10.222.0.65 () {60 vars in 1141 bytes} [Tue May 10 19:58:48 2022] POST /identity/v3/services => generated 241 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:58:48.405343 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc98101-ffb2-4b3a-96e2-50497061f4c2 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:48.487219 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe9b29-8516-478c-89b3-6aa4fcb4ed7b None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:48.528642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e2e3c-03f2-4208-a74f-bda00e967881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:48.529436 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e2e3c-03f2-4208-a74f-bda00e967881 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:48.529717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e2e3c-03f2-4208-a74f-bda00e967881 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:48.529991 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e2e3c-03f2-4208-a74f-bda00e967881 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:48.532424 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc98101-ffb2-4b3a-96e2-50497061f4c2 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:48.537592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e2e3c-03f2-4208-a74f-bda00e967881 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:48.539073 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e2e3c-03f2-4208-a74f-bda00e967881 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:48.554428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-2d5e2e3c-03f2-4208-a74f-bda00e967881 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:48.554428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:48.554428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:48.554428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:48.554428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:48.554428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:48.554428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:48.554428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:48.554428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:48.554428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:48.554428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:48.554428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:48.554428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:48.554428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:48.554428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:48.555856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:48.555856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:48.555856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:48.555856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:48.555856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:48.555856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:48.555856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:48.555856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:48.555856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:48.555856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:48.555856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:48.555856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:48.555856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:48.555856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:48.555856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:48.555856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:48.555856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:48.556742 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:48.556742 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:48.556742 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:48.556742 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:48.556742 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:48.556742 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:48.556742 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:48.556742 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:48.556742 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 458/920] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:48.563751 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e2e3c-03f2-4208-a74f-bda00e967881 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:48.576923 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53101d7c-d6a0-499b-9afc-e34fa86d3507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:48.578148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53101d7c-d6a0-499b-9afc-e34fa86d3507 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:48.578750 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53101d7c-d6a0-499b-9afc-e34fa86d3507 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:48.579206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53101d7c-d6a0-499b-9afc-e34fa86d3507 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:48.580109 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53101d7c-d6a0-499b-9afc-e34fa86d3507 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:48.580419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53101d7c-d6a0-499b-9afc-e34fa86d3507 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:48.581842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53101d7c-d6a0-499b-9afc-e34fa86d3507 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:48.587701 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 463/921] 10.222.0.65 () {56 vars in 1114 bytes} [Tue May 10 19:58:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:48.593190 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53101d7c-d6a0-499b-9afc-e34fa86d3507 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:48.681348 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e2e3c-03f2-4208-a74f-bda00e967881 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:48.704655 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53101d7c-d6a0-499b-9afc-e34fa86d3507 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:48.727400 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56409f-5f9a-4f9f-95ea-6af905415643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:48.728477 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56409f-5f9a-4f9f-95ea-6af905415643 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:48.728918 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56409f-5f9a-4f9f-95ea-6af905415643 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:48.729701 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56409f-5f9a-4f9f-95ea-6af905415643 None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:48.731018 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56409f-5f9a-4f9f-95ea-6af905415643 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:48.731667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56409f-5f9a-4f9f-95ea-6af905415643 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:48.733913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56409f-5f9a-4f9f-95ea-6af905415643 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:48.740989 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6f13e-cd5b-46af-a16f-302b871fdcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:48.741713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6f13e-cd5b-46af-a16f-302b871fdcd9 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:48.742107 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6f13e-cd5b-46af-a16f-302b871fdcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:48.742489 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6f13e-cd5b-46af-a16f-302b871fdcd9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:48.743339 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6f13e-cd5b-46af-a16f-302b871fdcd9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:48.744839 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6f13e-cd5b-46af-a16f-302b871fdcd9 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:48.749718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 459/922] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:48.756708 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56409f-5f9a-4f9f-95ea-6af905415643 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:48.838197 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 464/923] 10.222.0.65 () {60 vars in 1144 bytes} [Tue May 10 19:58:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 19:58:48.873858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56409f-5f9a-4f9f-95ea-6af905415643 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:48.918114 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aafd2d-9b30-4434-bac1-b913a3e94138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:48.918794 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aafd2d-9b30-4434-bac1-b913a3e94138 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:48.919080 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aafd2d-9b30-4434-bac1-b913a3e94138 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:48.919357 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aafd2d-9b30-4434-bac1-b913a3e94138 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:48.920203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aafd2d-9b30-4434-bac1-b913a3e94138 None admin] RBAC: Adding query filter params (role.id=2cb62fd2d3a747ee8827a5d19d6f9f2e, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=7ec2605fd1b84e0ea134b943ed33a605) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:48.920512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aafd2d-9b30-4434-bac1-b913a3e94138 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:48.922616 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aafd2d-9b30-4434-bac1-b913a3e94138 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:48.929325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 460/924] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:58:48 2022] GET /identity/v3/role_assignments?user.id=7ec2605fd1b84e0ea134b943ed33a605&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=2cb62fd2d3a747ee8827a5d19d6f9f2e => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:50.165716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f28bb-2a00-475a-9047-aefaa6d68507 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:50.166261 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f28bb-2a00-475a-9047-aefaa6d68507 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:50.166370 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f28bb-2a00-475a-9047-aefaa6d68507 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:50.167773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 465/925] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:50.173786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663807f-0649-4e99-a2c8-31a4c166058c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:50.174222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663807f-0649-4e99-a2c8-31a4c166058c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:50.174392 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663807f-0649-4e99-a2c8-31a4c166058c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:50.192790 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-2663807f-0649-4e99-a2c8-31a4c166058c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:50.194343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-2663807f-0649-4e99-a2c8-31a4c166058c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:50.262105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 461/926] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:50.283438 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f28bb-2a00-475a-9047-aefaa6d68507 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:50.387690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f28bb-2a00-475a-9047-aefaa6d68507 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:50.421681 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ff12e-eef4-41fa-979f-0a36402c21bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:50.423343 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ff12e-eef4-41fa-979f-0a36402c21bf None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:50.423810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ff12e-eef4-41fa-979f-0a36402c21bf None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:50.424266 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ff12e-eef4-41fa-979f-0a36402c21bf None admin] PATH_INFO: `/v3/services/compute` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:50.425120 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ff12e-eef4-41fa-979f-0a36402c21bf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:50.426698 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ff12e-eef4-41fa-979f-0a36402c21bf None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:50.438979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-b48ff12e-eef4-41fa-979f-0a36402c21bf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 10 19:58:50.438979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:50.438979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:50.438979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:50.438979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:50.438979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:50.438979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:50.438979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:50.438979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 10 19:58:50.438979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:50.438979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:50.438979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:50.438979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:50.438979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:50.438979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:50.438979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:50.438979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:50.439938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:50.439938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:50.439938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:50.439938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:50.439938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:50.439938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:50.439938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:50.439938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:50.439938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:50.439938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:50.439938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:50.439938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:50.439938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:50.439938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:50.439938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:50.439938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:50.440951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:50.440951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:50.440951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:50.440951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:50.440951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:50.440951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:50.440951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:50.440951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:50.440951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:50.440951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:50.440951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:50.440951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:50.440951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 10 19:58:50.440951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:50.440951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 466/927] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:50.446460 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663807f-0649-4e99-a2c8-31a4c166058c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:50.544649 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663807f-0649-4e99-a2c8-31a4c166058c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:50.581472 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075de084-72e3-4397-b489-ca76a1f6fc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:50.582173 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075de084-72e3-4397-b489-ca76a1f6fc67 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:50.582461 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075de084-72e3-4397-b489-ca76a1f6fc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:50.582738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075de084-72e3-4397-b489-ca76a1f6fc67 None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:50.583461 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075de084-72e3-4397-b489-ca76a1f6fc67 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:50.583771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075de084-72e3-4397-b489-ca76a1f6fc67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:50.585140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075de084-72e3-4397-b489-ca76a1f6fc67 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:50.593335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 462/928] 10.222.0.65 () {56 vars in 1112 bytes} [Tue May 10 19:58:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:50.599191 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ff12e-eef4-41fa-979f-0a36402c21bf None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:50.701672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ff12e-eef4-41fa-979f-0a36402c21bf None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:50.735333 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2403f2-4780-4d31-bc05-9cb1552c15a8 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:50.735620 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2403f2-4780-4d31-bc05-9cb1552c15a8 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:50.735904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2403f2-4780-4d31-bc05-9cb1552c15a8 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:50.736115 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce8952-ace0-4c30-b7be-2a293023302c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:50.736773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce8952-ace0-4c30-b7be-2a293023302c None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:50.737054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce8952-ace0-4c30-b7be-2a293023302c None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:50.737325 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce8952-ace0-4c30-b7be-2a293023302c None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:50.737546 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 463/929] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:50.738332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce8952-ace0-4c30-b7be-2a293023302c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:50.738640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce8952-ace0-4c30-b7be-2a293023302c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:50.739992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce8952-ace0-4c30-b7be-2a293023302c None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:50.742984 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2f61b-9ae9-4859-8227-5f9ad004f3b6 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:50.743279 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2f61b-9ae9-4859-8227-5f9ad004f3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:50.743559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2f61b-9ae9-4859-8227-5f9ad004f3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:50.744958 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 467/930] 10.222.0.65 () {56 vars in 1112 bytes} [Tue May 10 19:58:50 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:50.767056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-89e2f61b-9ae9-4859-8227-5f9ad004f3b6 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:50.768262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-89e2f61b-9ae9-4859-8227-5f9ad004f3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:50.843621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 464/931] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:58:50.866481 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce8952-ace0-4c30-b7be-2a293023302c None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:50.976667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce8952-ace0-4c30-b7be-2a293023302c None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:51.011341 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ad409-6e51-4a84-9ff2-dcb9fff3f825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:51.012151 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ad409-6e51-4a84-9ff2-dcb9fff3f825 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:51.012437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ad409-6e51-4a84-9ff2-dcb9fff3f825 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:51.012716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ad409-6e51-4a84-9ff2-dcb9fff3f825 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:51.013389 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ad409-6e51-4a84-9ff2-dcb9fff3f825 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:51.014782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ad409-6e51-4a84-9ff2-dcb9fff3f825 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:51.026962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-958ad409-6e51-4a84-9ff2-dcb9fff3f825 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 19:58:51.026962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:51.026962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:51.026962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:51.026962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:51.026962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:51.026962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:51.026962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:51.026962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 19:58:51.026962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:51.026962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:51.026962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:51.026962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:51.026962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:51.026962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:51.026962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:51.026962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:51.027951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:51.027951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:51.027951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:51.027951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:51.027951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:51.027951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:51.027951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:51.027951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:51.027951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:51.027951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:51.027951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:51.027951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:51.027951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:51.027951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:51.027951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:51.027951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:51.029229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:51.029229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:51.029229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:51.029229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:51.029229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:51.029229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:51.029229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:51.029229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:51.029229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:51.029229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:51.029229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:51.029229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:51.029229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 19:58:51.029229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:51.029229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 468/932] 10.222.0.65 () {56 vars in 1126 bytes} [Tue May 10 19:58:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:51.034463 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2f61b-9ae9-4859-8227-5f9ad004f3b6 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:51.119323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f8c13-ab61-4c91-9ad2-5bcca1fead59 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:51.119323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f8c13-ab61-4c91-9ad2-5bcca1fead59 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:51.119609 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f8c13-ab61-4c91-9ad2-5bcca1fead59 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:51.121160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 469/933] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:58:51.126274 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f10e9-bdfa-4d20-986c-a53aadf7a08a None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:51.126563 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f10e9-bdfa-4d20-986c-a53aadf7a08a None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:51.126847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f10e9-bdfa-4d20-986c-a53aadf7a08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:51.141667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2f61b-9ae9-4859-8227-5f9ad004f3b6 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:51.146863 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-ce6f10e9-bdfa-4d20-986c-a53aadf7a08a None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:51.148234 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-ce6f10e9-bdfa-4d20-986c-a53aadf7a08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:51.177376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14716aa1-db4b-415a-b365-9fa1a7407f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:51.178062 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14716aa1-db4b-415a-b365-9fa1a7407f1f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:51.178442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14716aa1-db4b-415a-b365-9fa1a7407f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:51.178626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14716aa1-db4b-415a-b365-9fa1a7407f1f None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:51.179296 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14716aa1-db4b-415a-b365-9fa1a7407f1f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:51.179603 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14716aa1-db4b-415a-b365-9fa1a7407f1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:51.180967 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14716aa1-db4b-415a-b365-9fa1a7407f1f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:51.186254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 465/934] 10.222.0.65 () {56 vars in 1122 bytes} [Tue May 10 19:58:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:51.191748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14716aa1-db4b-415a-b365-9fa1a7407f1f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:51.238246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 470/935] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:58:51.255531 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f10e9-bdfa-4d20-986c-a53aadf7a08a None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:51.293040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14716aa1-db4b-415a-b365-9fa1a7407f1f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:51.327526 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e323101-1f9c-49fb-a4a3-01837a3db207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:51.328231 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e323101-1f9c-49fb-a4a3-01837a3db207 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:51.328514 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e323101-1f9c-49fb-a4a3-01837a3db207 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:51.328795 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e323101-1f9c-49fb-a4a3-01837a3db207 None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:51.329518 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e323101-1f9c-49fb-a4a3-01837a3db207 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:51.329829 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e323101-1f9c-49fb-a4a3-01837a3db207 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:51.331231 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e323101-1f9c-49fb-a4a3-01837a3db207 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:51.340605 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 466/936] 10.222.0.65 () {56 vars in 1122 bytes} [Tue May 10 19:58:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:51.340605 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc4b81-0f27-4856-b68f-38bfeb92115f None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:51.340605 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc4b81-0f27-4856-b68f-38bfeb92115f None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:51.340605 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc4b81-0f27-4856-b68f-38bfeb92115f None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:51.340605 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 467/937] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:51.343285 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc4b81-0f27-4856-b68f-38bfeb92115f None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:51.364847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f10e9-bdfa-4d20-986c-a53aadf7a08a None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:51.402440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0220aa-6146-4280-9537-f7baa71b97f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:51.403101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0220aa-6146-4280-9537-f7baa71b97f9 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:51.403384 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0220aa-6146-4280-9537-f7baa71b97f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:51.403663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0220aa-6146-4280-9537-f7baa71b97f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:51.404365 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0220aa-6146-4280-9537-f7baa71b97f9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:51.405791 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0220aa-6146-4280-9537-f7baa71b97f9 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:51.407707 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: WARNING keystone.server.flask.common [None req-0b0220aa-6146-4280-9537-f7baa71b97f9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 10 19:58:51.452654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc4b81-0f27-4856-b68f-38bfeb92115f None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:51.460931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.sql.core [None req-0b0220aa-6146-4280-9537-f7baa71b97f9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 10 19:58:51.460931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 10 19:58:51.460931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [parameters: {'id': '287a0452c1544058afe306a2b426bbad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 10 19:58:51.460931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80516) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 10 19:58:51.468497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-0b0220aa-6146-4280-9537-f7baa71b97f9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 287a0452c1544058afe306a2b426bbad.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 287a0452c1544058afe306a2b426bbad. May 10 19:58:51.468497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:51.468497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 10 19:58:51.468497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self.dialect.do_execute( May 10 19:58:51.468497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 10 19:58:51.468497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 10 19:58:51.468497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 10 19:58:51.468497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application result = self._query(query) May 10 19:58:51.468497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 10 19:58:51.468497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application conn.query(q) May 10 19:58:51.468497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 10 19:58:51.468497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 10 19:58:51.468497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 10 19:58:51.470013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application result.read() May 10 19:58:51.470013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 10 19:58:51.470013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 10 19:58:51.470013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 10 19:58:51.470013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application packet.raise_for_error() May 10 19:58:51.470013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 10 19:58:51.470013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 10 19:58:51.470013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 10 19:58:51.470013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 10 19:58:51.470013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 10 19:58:51.470013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:51.470013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 10 19:58:51.470013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:51.470013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:51.470013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 10 19:58:51.470013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 10 19:58:51.470013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 10 19:58:51.470013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return project_ref.to_dict() May 10 19:58:51.471271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 10 19:58:51.471271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application next(self.gen) May 10 19:58:51.471271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 10 19:58:51.471271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application yield resource May 10 19:58:51.471271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 10 19:58:51.471271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application next(self.gen) May 10 19:58:51.471271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 10 19:58:51.471271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self.session.rollback() May 10 19:58:51.471271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 10 19:58:51.471271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self.force_reraise() May 10 19:58:51.471271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 10 19:58:51.471271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise self.value May 10 19:58:51.471271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 10 19:58:51.471271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 10 19:58:51.471271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 10 19:58:51.471271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application session.commit() May 10 19:58:51.471271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 10 19:58:51.472217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 10 19:58:51.472217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 10 19:58:51.472217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._prepare_impl() May 10 19:58:51.472217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 10 19:58:51.472217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self.session.flush() May 10 19:58:51.472217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 10 19:58:51.472217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._flush(objects) May 10 19:58:51.472217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 10 19:58:51.472217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 10 19:58:51.472217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 10 19:58:51.472217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application compat.raise_( May 10 19:58:51.472217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 10 19:58:51.472217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception May 10 19:58:51.472217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 10 19:58:51.472217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application flush_context.execute() May 10 19:58:51.472217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 10 19:58:51.472217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rec.execute(self) May 10 19:58:51.474596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 10 19:58:51.474596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 10 19:58:51.474596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 10 19:58:51.474596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application _emit_insert_statements( May 10 19:58:51.474596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 10 19:58:51.474596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application c = connection._execute_20( May 10 19:58:51.474596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 10 19:58:51.474596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 10 19:58:51.474596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 10 19:58:51.474596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 10 19:58:51.474596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 10 19:58:51.474596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ret = self._execute_context( May 10 19:58:51.474596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 10 19:58:51.474596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 10 19:58:51.474596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 10 19:58:51.477158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 10 19:58:51.477158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 10 19:58:51.477158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception May 10 19:58:51.477158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 10 19:58:51.477158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self.dialect.do_execute( May 10 19:58:51.477158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 10 19:58:51.477158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 10 19:58:51.477158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 10 19:58:51.477158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application result = self._query(query) May 10 19:58:51.477158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 10 19:58:51.477158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application conn.query(q) May 10 19:58:51.477158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 10 19:58:51.477158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 10 19:58:51.477158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 10 19:58:51.477158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application result.read() May 10 19:58:51.477158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 10 19:58:51.477158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 10 19:58:51.481209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 10 19:58:51.481209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application packet.raise_for_error() May 10 19:58:51.481209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 10 19:58:51.481209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 10 19:58:51.481209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 10 19:58:51.481209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 10 19:58:51.481209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 10 19:58:51.481209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 10 19:58:51.481209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [parameters: {'id': '287a0452c1544058afe306a2b426bbad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 10 19:58:51.481209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 10 19:58:51.481209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:51.481209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:51.481209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:51.481209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:51.481209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 10 19:58:51.483440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 10 19:58:51.483440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 10 19:58:51.483440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 10 19:58:51.483440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 10 19:58:51.483440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:51.483440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:51.483440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:51.483440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:51.483440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:51.483440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:51.483440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:51.483440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:51.483440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:51.483440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:51.483440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:51.483440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:51.483440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:51.487175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:51.487175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 10 19:58:51.487175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 10 19:58:51.487175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:51.487175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:51.487175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 10 19:58:51.487175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.Conflict( May 10 19:58:51.487175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 287a0452c1544058afe306a2b426bbad. May 10 19:58:51.487175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:51.487175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 471/938] 10.222.0.65 () {60 vars in 1140 bytes} [Tue May 10 19:58:51 2022] POST /identity/v3/projects => generated 262 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:51.487175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ec630-7b19-48d4-94c5-7ba0ee6a96e3 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:51.487175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ec630-7b19-48d4-94c5-7ba0ee6a96e3 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:51.489671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ec630-7b19-48d4-94c5-7ba0ee6a96e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:51.491024 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-fc8ec630-7b19-48d4-94c5-7ba0ee6a96e3 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:51.492348 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-fc8ec630-7b19-48d4-94c5-7ba0ee6a96e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:51.495662 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d67971-0f2b-4ca8-8b3b-bc9f9a6e9932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:51.496324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d67971-0f2b-4ca8-8b3b-bc9f9a6e9932 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:51.496609 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d67971-0f2b-4ca8-8b3b-bc9f9a6e9932 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:51.496888 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d67971-0f2b-4ca8-8b3b-bc9f9a6e9932 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:51.497687 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d67971-0f2b-4ca8-8b3b-bc9f9a6e9932 None admin] RBAC: Adding query filter params (interface=public, service_id=973d17a2b54144c7a75c5f89f5189a5c, region_id=RegionOne) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:51.498003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d67971-0f2b-4ca8-8b3b-bc9f9a6e9932 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:51.499414 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d67971-0f2b-4ca8-8b3b-bc9f9a6e9932 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:51.504518 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 468/939] 10.222.0.65 () {56 vars in 1251 bytes} [Tue May 10 19:58:51 2022] GET /identity/v3/endpoints?service_id=973d17a2b54144c7a75c5f89f5189a5c&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:51.504518 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d67971-0f2b-4ca8-8b3b-bc9f9a6e9932 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:51.563715 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 472/940] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:51.564591 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ec630-7b19-48d4-94c5-7ba0ee6a96e3 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:51.606218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d67971-0f2b-4ca8-8b3b-bc9f9a6e9932 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:51.641093 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a092e8-638d-48de-87e5-25b2fdc7743f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:51.642216 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a092e8-638d-48de-87e5-25b2fdc7743f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:51.642565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a092e8-638d-48de-87e5-25b2fdc7743f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:51.642978 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a092e8-638d-48de-87e5-25b2fdc7743f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:51.650501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a092e8-638d-48de-87e5-25b2fdc7743f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:51.652018 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a092e8-638d-48de-87e5-25b2fdc7743f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:51.666281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-57a092e8-638d-48de-87e5-25b2fdc7743f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:51.666281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:51.666281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:51.666281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:51.666281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:51.666281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:51.666281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:51.666281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:51.666281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:51.666281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:51.666281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:51.666281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:51.666281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:51.666281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:51.666281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:51.667368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:51.667368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:51.667368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:51.667368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:51.667368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:51.667368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:51.667368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:51.667368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:51.667368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:51.667368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:51.667368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:51.667368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:51.667368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:51.667368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:51.667368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:51.667368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:51.667368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:51.668483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:51.668483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:51.668483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:51.668483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:51.668483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:51.668483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:51.668483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:51.668483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:51.670164 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 469/941] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:51.670857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a092e8-638d-48de-87e5-25b2fdc7743f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:51.680090 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ec630-7b19-48d4-94c5-7ba0ee6a96e3 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:51.718014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a82ffa-5828-49fb-9531-84ac3be38bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:51.718750 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a82ffa-5828-49fb-9531-84ac3be38bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:51.719036 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a82ffa-5828-49fb-9531-84ac3be38bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:51.719310 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a82ffa-5828-49fb-9531-84ac3be38bd4 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:51.719997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a82ffa-5828-49fb-9531-84ac3be38bd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:51.721563 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a82ffa-5828-49fb-9531-84ac3be38bd4 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:51.730033 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 473/942] 10.222.0.65 () {56 vars in 1087 bytes} [Tue May 10 19:58:51 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 19:58:51.730819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a82ffa-5828-49fb-9531-84ac3be38bd4 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:51.793754 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a092e8-638d-48de-87e5-25b2fdc7743f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:51.832050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22e0ae-d4e9-4627-acba-52030816290d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:51.833244 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22e0ae-d4e9-4627-acba-52030816290d None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:51.833736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22e0ae-d4e9-4627-acba-52030816290d None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:51.834287 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22e0ae-d4e9-4627-acba-52030816290d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:51.846447 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22e0ae-d4e9-4627-acba-52030816290d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:51.854474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a82ffa-5828-49fb-9531-84ac3be38bd4 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:51.855641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-4e22e0ae-d4e9-4627-acba-52030816290d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 19:58:51.855641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:51.855641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:51.855641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:51.855641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:51.855641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:51.855641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:51.855641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:51.855641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:51.855641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:51.855641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:51.855641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:51.855641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:51.855641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:51.855641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:51.855641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:51.856551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:51.856551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:51.856551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:51.856551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:51.856551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:51.856551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:51.856551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:51.856551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:51.856551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:51.856551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:51.856551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:51.856551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:51.856551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:51.856551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:51.856551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:51.856551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:51.857384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:51.857384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:51.857384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:51.857384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:51.857384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:51.857384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:51.857384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:51.857384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:51.857384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:51.857384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:51.857384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 19:58:51.857384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:51.865638 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 470/943] 10.222.0.65 () {56 vars in 1120 bytes} [Tue May 10 19:58:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:51.867047 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22e0ae-d4e9-4627-acba-52030816290d None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:51.891752 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043af1c9-e9eb-49c3-9f62-44a1c99fe2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:51.892429 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043af1c9-e9eb-49c3-9f62-44a1c99fe2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:51.892714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043af1c9-e9eb-49c3-9f62-44a1c99fe2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:51.892989 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043af1c9-e9eb-49c3-9f62-44a1c99fe2e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:51.893843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043af1c9-e9eb-49c3-9f62-44a1c99fe2e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:51.894182 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043af1c9-e9eb-49c3-9f62-44a1c99fe2e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:51.895753 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043af1c9-e9eb-49c3-9f62-44a1c99fe2e3 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:51.915889 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 474/944] 10.222.0.65 () {56 vars in 1112 bytes} [Tue May 10 19:58:51 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:51.986417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22e0ae-d4e9-4627-acba-52030816290d None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:52.023704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc06c0b-9900-400d-bb70-ae1d88f3016d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:52.024541 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc06c0b-9900-400d-bb70-ae1d88f3016d None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:52.024929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc06c0b-9900-400d-bb70-ae1d88f3016d None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:52.025310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc06c0b-9900-400d-bb70-ae1d88f3016d None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:52.026220 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc06c0b-9900-400d-bb70-ae1d88f3016d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:52.026631 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc06c0b-9900-400d-bb70-ae1d88f3016d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:52.028131 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc06c0b-9900-400d-bb70-ae1d88f3016d None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:52.038375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 471/945] 10.222.0.65 () {56 vars in 1115 bytes} [Tue May 10 19:58:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:52.044401 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043af1c9-e9eb-49c3-9f62-44a1c99fe2e3 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:52.150907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043af1c9-e9eb-49c3-9f62-44a1c99fe2e3 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:52.185736 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e246de-76a0-4847-8634-f8e999a7abea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:52.186576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e246de-76a0-4847-8634-f8e999a7abea None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:52.186863 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e246de-76a0-4847-8634-f8e999a7abea None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:52.187144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e246de-76a0-4847-8634-f8e999a7abea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:52.197701 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e246de-76a0-4847-8634-f8e999a7abea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:52.199200 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e246de-76a0-4847-8634-f8e999a7abea None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:52.217549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-71e246de-76a0-4847-8634-f8e999a7abea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:52.217549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:52.217549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:52.217549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:52.217549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:52.217549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:52.217549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:52.217549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:52.217549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:52.217549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:52.217549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:52.217549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:52.217549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:52.217549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:52.217549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:52.218487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:52.218487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:52.218487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:52.218487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:52.218487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:52.218487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:52.218487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:52.218487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:52.218487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:52.218487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:52.218487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:52.218487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:52.218487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:52.218487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:52.218487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:52.218487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:52.218487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:52.219966 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:52.219966 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:52.219966 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:52.219966 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:52.219966 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 475/946] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:52.226309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc06c0b-9900-400d-bb70-ae1d88f3016d None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:52.325422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc06c0b-9900-400d-bb70-ae1d88f3016d None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:52.361887 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ba32a-6b51-4a6d-baba-26f20f47dc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:52.362842 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ba32a-6b51-4a6d-baba-26f20f47dc58 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:52.363122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ba32a-6b51-4a6d-baba-26f20f47dc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:52.363395 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ba32a-6b51-4a6d-baba-26f20f47dc58 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:52.364175 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ba32a-6b51-4a6d-baba-26f20f47dc58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:52.364478 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ba32a-6b51-4a6d-baba-26f20f47dc58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:52.365840 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ba32a-6b51-4a6d-baba-26f20f47dc58 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:52.378321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 472/947] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:52.383703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e246de-76a0-4847-8634-f8e999a7abea None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:52.489365 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e246de-76a0-4847-8634-f8e999a7abea None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:52.523603 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573af72-0e81-4779-9f08-e509de25430e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:52.524835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573af72-0e81-4779-9f08-e509de25430e None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:52.525322 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573af72-0e81-4779-9f08-e509de25430e None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:52.525800 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573af72-0e81-4779-9f08-e509de25430e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:52.533892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573af72-0e81-4779-9f08-e509de25430e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:52.535606 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573af72-0e81-4779-9f08-e509de25430e None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:52.551134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-9573af72-0e81-4779-9f08-e509de25430e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 19:58:52.551134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:52.551134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:52.551134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:52.551134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:52.551134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:52.551134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:52.551134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:52.551134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:52.551134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:52.551134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:52.551134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:52.551134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:52.551134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:52.551134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:52.551134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:52.552106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:52.552106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:52.552106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:52.552106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:52.552106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:52.552106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:52.552106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:52.552106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:52.552106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:52.552106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:52.552106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:52.552106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:52.552106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:52.552106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:52.552106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:52.552106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:52.552106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:52.552106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:52.553055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:52.553055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:52.553055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:52.553055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:52.553055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:52.553055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:52.553055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:52.553055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:52.553055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 19:58:52.553055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:52.555767 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 476/948] 10.222.0.65 () {56 vars in 1117 bytes} [Tue May 10 19:58:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:52.562499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ba32a-6b51-4a6d-baba-26f20f47dc58 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:52.665124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ba32a-6b51-4a6d-baba-26f20f47dc58 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:52.704209 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95758b-6f4d-41cd-a493-ba67904647f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:52.705289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95758b-6f4d-41cd-a493-ba67904647f6 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:52.705780 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95758b-6f4d-41cd-a493-ba67904647f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:52.706286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95758b-6f4d-41cd-a493-ba67904647f6 None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:52.707408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95758b-6f4d-41cd-a493-ba67904647f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:52.707936 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95758b-6f4d-41cd-a493-ba67904647f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:52.709846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95758b-6f4d-41cd-a493-ba67904647f6 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:52.728686 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 473/949] 10.222.0.65 () {56 vars in 1149 bytes} [Tue May 10 19:58:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:52.735404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573af72-0e81-4779-9f08-e509de25430e None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:52.838283 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573af72-0e81-4779-9f08-e509de25430e None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:52.872489 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c260c-f3f2-4b13-8929-4c8734d684e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:52.873277 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c260c-f3f2-4b13-8929-4c8734d684e7 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:52.873561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c260c-f3f2-4b13-8929-4c8734d684e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:52.873839 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c260c-f3f2-4b13-8929-4c8734d684e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:52.884596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c260c-f3f2-4b13-8929-4c8734d684e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:52.886051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c260c-f3f2-4b13-8929-4c8734d684e7 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:52.906022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-9c9c260c-f3f2-4b13-8929-4c8734d684e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:52.906022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:52.906022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:52.906022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:52.906022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:52.906022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:52.906022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:52.906022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:52.906022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:52.906022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:52.906022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:52.906022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:52.906022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:52.906022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:52.906022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:52.907052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:52.907052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:52.907052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:52.907052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:52.907052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:52.907052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:52.907052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:52.907052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:52.907052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:52.907052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:52.907052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:52.907052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:52.907052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:52.907052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:52.907052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:52.907052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:52.907052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:52.907973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:52.907973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:52.907973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:52.907973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:52.907973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 477/950] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:52.914485 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95758b-6f4d-41cd-a493-ba67904647f6 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:53.024164 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95758b-6f4d-41cd-a493-ba67904647f6 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:53.058669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866a81d-f315-41ac-aa33-3025fe6b5162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:53.059578 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866a81d-f315-41ac-aa33-3025fe6b5162 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:53.060009 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866a81d-f315-41ac-aa33-3025fe6b5162 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:53.060454 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866a81d-f315-41ac-aa33-3025fe6b5162 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:53.061327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866a81d-f315-41ac-aa33-3025fe6b5162 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:53.061777 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866a81d-f315-41ac-aa33-3025fe6b5162 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:53.063930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866a81d-f315-41ac-aa33-3025fe6b5162 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:53.076215 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 474/951] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:53.086129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c260c-f3f2-4b13-8929-4c8734d684e7 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:53.098695 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c67cf7-d401-4ca7-b89e-4d470433e8d0 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:53.099119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c67cf7-d401-4ca7-b89e-4d470433e8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:53.099499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c67cf7-d401-4ca7-b89e-4d470433e8d0 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:53.101224 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 475/952] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:58:53.106483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefdea3-1f03-4301-a079-3bfb3d287233 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:53.106881 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefdea3-1f03-4301-a079-3bfb3d287233 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:53.107416 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefdea3-1f03-4301-a079-3bfb3d287233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:53.132534 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-acefdea3-1f03-4301-a079-3bfb3d287233 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:53.134116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-acefdea3-1f03-4301-a079-3bfb3d287233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:53.189712 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c260c-f3f2-4b13-8929-4c8734d684e7 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:53.213136 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 476/953] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:53.236254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefdea3-1f03-4301-a079-3bfb3d287233 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:53.238481 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1fe020-8d9e-41a6-98a8-63cd3eb987c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:53.239583 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fe020-8d9e-41a6-98a8-63cd3eb987c6 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:53.240105 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fe020-8d9e-41a6-98a8-63cd3eb987c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:53.240644 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fe020-8d9e-41a6-98a8-63cd3eb987c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:53.248604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1fe020-8d9e-41a6-98a8-63cd3eb987c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:53.251132 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1fe020-8d9e-41a6-98a8-63cd3eb987c6 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:53.265464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-ee1fe020-8d9e-41a6-98a8-63cd3eb987c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:53.265464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:53.265464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:53.265464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:53.265464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:53.265464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:53.265464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:53.265464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:53.265464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:53.265464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:53.265464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:53.265464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:53.265464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:53.265464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:53.265464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:53.269114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:53.269114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:53.269114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:53.269114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:53.269114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:53.269114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:53.269114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:53.269114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:53.269114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:53.269114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:53.269114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:53.269114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:53.269114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:53.269114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:53.269114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:53.269114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:53.269114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:53.270046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:53.270046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:53.270046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:53.270046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:53.270046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:53.270046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:53.270046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:53.270046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:53.270046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 478/954] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:53.273279 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1fe020-8d9e-41a6-98a8-63cd3eb987c6 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:53.351994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefdea3-1f03-4301-a079-3bfb3d287233 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:53.375172 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1fe020-8d9e-41a6-98a8-63cd3eb987c6 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:53.393561 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e237849-78c9-4754-9d98-93b755c82720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:53.394276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e237849-78c9-4754-9d98-93b755c82720 None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:53.394993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e237849-78c9-4754-9d98-93b755c82720 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:53.395270 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e237849-78c9-4754-9d98-93b755c82720 None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:53.396006 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e237849-78c9-4754-9d98-93b755c82720 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:53.397731 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e237849-78c9-4754-9d98-93b755c82720 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:53.428449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25ce10-28e2-4ac0-b51f-81c5abda9652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:53.429139 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25ce10-28e2-4ac0-b51f-81c5abda9652 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:53.429401 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25ce10-28e2-4ac0-b51f-81c5abda9652 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:53.429680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25ce10-28e2-4ac0-b51f-81c5abda9652 None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:53.430574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25ce10-28e2-4ac0-b51f-81c5abda9652 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:53.430885 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25ce10-28e2-4ac0-b51f-81c5abda9652 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:53.432319 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25ce10-28e2-4ac0-b51f-81c5abda9652 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:53.445241 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 479/955] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:53.451012 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25ce10-28e2-4ac0-b51f-81c5abda9652 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:53.483847 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 477/956] 10.222.0.65 () {60 vars in 1141 bytes} [Tue May 10 19:58:53 2022] POST /identity/v3/services => generated 242 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 19:58:53.564399 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25ce10-28e2-4ac0-b51f-81c5abda9652 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:53.611041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cda721-acc4-4253-9573-a53bc73fe946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:53.612210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cda721-acc4-4253-9573-a53bc73fe946 None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:53.612901 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cda721-acc4-4253-9573-a53bc73fe946 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:53.613319 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cda721-acc4-4253-9573-a53bc73fe946 None admin] PATH_INFO: `/v3/projects/a006477d1df94390b599621446bd0f02/users/7ec2605fd1b84e0ea134b943ed33a605/roles/2cb62fd2d3a747ee8827a5d19d6f9f2e` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:53.615830 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cda721-acc4-4253-9573-a53bc73fe946 None admin] RBAC: Authorizing `identity:create_grant(project_id=a006477d1df94390b599621446bd0f02, user_id=7ec2605fd1b84e0ea134b943ed33a605, role_id=2cb62fd2d3a747ee8827a5d19d6f9f2e)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:53.617848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cda721-acc4-4253-9573-a53bc73fe946 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:53.715168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 480/957] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:58:53 2022] PUT /identity/v3/projects/a006477d1df94390b599621446bd0f02/users/7ec2605fd1b84e0ea134b943ed33a605/roles/2cb62fd2d3a747ee8827a5d19d6f9f2e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 19:58:54.160703 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0095d-c03b-40c5-8ab7-b766f76eee54 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:54.161073 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0095d-c03b-40c5-8ab7-b766f76eee54 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:54.161382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0095d-c03b-40c5-8ab7-b766f76eee54 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:54.163056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 478/958] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:54.168399 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0656c9-75a2-45cf-9339-0101bea30c3f None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:54.168693 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0656c9-75a2-45cf-9339-0101bea30c3f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:54.168967 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0656c9-75a2-45cf-9339-0101bea30c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:54.185526 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-4f0656c9-75a2-45cf-9339-0101bea30c3f None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:54.186744 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-4f0656c9-75a2-45cf-9339-0101bea30c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:54.259955 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 481/959] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:54.267230 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cfebd-8de9-4410-8574-2b6ce6516e57 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:54.267702 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cfebd-8de9-4410-8574-2b6ce6516e57 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:54.267795 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cfebd-8de9-4410-8574-2b6ce6516e57 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:54.281812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 479/960] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:54.281946 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94aa46-fa4d-43e2-8285-bc198858b8f1 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:54.281946 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94aa46-fa4d-43e2-8285-bc198858b8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:54.281946 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94aa46-fa4d-43e2-8285-bc198858b8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:54.283495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cfebd-8de9-4410-8574-2b6ce6516e57 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:54.293145 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-ca94aa46-fa4d-43e2-8285-bc198858b8f1 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:54.294443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-ca94aa46-fa4d-43e2-8285-bc198858b8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:54.370199 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 482/961] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:54.388127 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca94aa46-fa4d-43e2-8285-bc198858b8f1 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:54.389939 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cfebd-8de9-4410-8574-2b6ce6516e57 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:54.425422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f596a-fa75-4b09-8fc9-dae442043375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:54.426258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f596a-fa75-4b09-8fc9-dae442043375 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:54.426599 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f596a-fa75-4b09-8fc9-dae442043375 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:54.426936 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f596a-fa75-4b09-8fc9-dae442043375 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:54.428099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f596a-fa75-4b09-8fc9-dae442043375 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:54.429908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f596a-fa75-4b09-8fc9-dae442043375 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:54.445712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-b82f596a-fa75-4b09-8fc9-dae442043375 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 19:58:54.445712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:54.445712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:54.445712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:54.445712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:54.445712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:54.445712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:54.445712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:54.445712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 19:58:54.445712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:54.445712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:54.445712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:54.445712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:54.445712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:54.445712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:54.445712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:54.445712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:54.447051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:54.447051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:54.447051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:54.447051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:54.447051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:54.447051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:54.447051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:54.447051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:54.447051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:54.447051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:54.447051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:54.447051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:54.447051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:54.447051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:54.447051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:54.447051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:54.448079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:54.448079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:54.448079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:54.448079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:54.448079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:54.448079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:54.448079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:54.448079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:54.448079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:54.448079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:54.448079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:54.448079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:54.448079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 19:58:54.448079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:54.450161 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 480/962] 10.222.0.65 () {56 vars in 1126 bytes} [Tue May 10 19:58:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:54.456665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f596a-fa75-4b09-8fc9-dae442043375 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:54.500592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca94aa46-fa4d-43e2-8285-bc198858b8f1 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:54.534296 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fb245-7d9e-4e0c-87f3-b45a3e2866e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:54.534962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fb245-7d9e-4e0c-87f3-b45a3e2866e0 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:54.535243 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fb245-7d9e-4e0c-87f3-b45a3e2866e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:54.535516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fb245-7d9e-4e0c-87f3-b45a3e2866e0 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:54.536225 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fb245-7d9e-4e0c-87f3-b45a3e2866e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:54.537650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fb245-7d9e-4e0c-87f3-b45a3e2866e0 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:54.559978 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f596a-fa75-4b09-8fc9-dae442043375 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:54.575414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.sql.core [None req-620fb245-7d9e-4e0c-87f3-b45a3e2866e0 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 10 19:58:54.575414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 10 19:58:54.575414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [parameters: {'user_id': '65a16ae92abc46e19c9cb787cd1cc905', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 10 19:58:54.575414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80516) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 10 19:58:54.586076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-620fb245-7d9e-4e0c-87f3-b45a3e2866e0 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 10 19:58:54.586076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:54.586076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 10 19:58:54.586076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self.dialect.do_execute( May 10 19:58:54.586076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 10 19:58:54.586076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 10 19:58:54.586076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 10 19:58:54.586076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application result = self._query(query) May 10 19:58:54.586076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 10 19:58:54.586076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application conn.query(q) May 10 19:58:54.586076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 10 19:58:54.586076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 10 19:58:54.586076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 10 19:58:54.586076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application result.read() May 10 19:58:54.586076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 10 19:58:54.587031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 10 19:58:54.587031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 10 19:58:54.587031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application packet.raise_for_error() May 10 19:58:54.587031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 10 19:58:54.587031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 10 19:58:54.587031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 10 19:58:54.587031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 10 19:58:54.587031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 10 19:58:54.587031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:54.587031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 10 19:58:54.587031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:54.587031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:54.587031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 10 19:58:54.587031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 10 19:58:54.587031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 10 19:58:54.587031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 10 19:58:54.587031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 10 19:58:54.587031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application next(self.gen) May 10 19:58:54.587953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 10 19:58:54.587953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application yield resource May 10 19:58:54.587953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 10 19:58:54.587953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application next(self.gen) May 10 19:58:54.587953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 10 19:58:54.587953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self.session.rollback() May 10 19:58:54.587953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 10 19:58:54.587953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self.force_reraise() May 10 19:58:54.587953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 10 19:58:54.587953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise self.value May 10 19:58:54.587953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 10 19:58:54.587953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 10 19:58:54.587953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 10 19:58:54.587953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application session.commit() May 10 19:58:54.587953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 10 19:58:54.587953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 10 19:58:54.587953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 10 19:58:54.588822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._prepare_impl() May 10 19:58:54.588822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 10 19:58:54.588822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self.session.flush() May 10 19:58:54.588822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 10 19:58:54.588822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._flush(objects) May 10 19:58:54.588822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 10 19:58:54.588822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 10 19:58:54.588822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 10 19:58:54.588822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application compat.raise_( May 10 19:58:54.588822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 10 19:58:54.588822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception May 10 19:58:54.588822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 10 19:58:54.588822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application flush_context.execute() May 10 19:58:54.588822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 10 19:58:54.588822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rec.execute(self) May 10 19:58:54.588822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 10 19:58:54.588822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 10 19:58:54.589919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 10 19:58:54.589919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application _emit_insert_statements( May 10 19:58:54.589919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 10 19:58:54.589919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application result = connection._execute_20( May 10 19:58:54.589919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 10 19:58:54.589919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 10 19:58:54.589919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 10 19:58:54.589919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 10 19:58:54.589919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 10 19:58:54.589919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ret = self._execute_context( May 10 19:58:54.589919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 10 19:58:54.589919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 10 19:58:54.589919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 10 19:58:54.589919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 10 19:58:54.589919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 10 19:58:54.589919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception May 10 19:58:54.591699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 10 19:58:54.591699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self.dialect.do_execute( May 10 19:58:54.591699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 10 19:58:54.591699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 10 19:58:54.591699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 10 19:58:54.591699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application result = self._query(query) May 10 19:58:54.591699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 10 19:58:54.591699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application conn.query(q) May 10 19:58:54.591699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 10 19:58:54.591699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 10 19:58:54.591699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 10 19:58:54.591699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application result.read() May 10 19:58:54.591699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 10 19:58:54.591699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 10 19:58:54.591699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 10 19:58:54.591699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application packet.raise_for_error() May 10 19:58:54.593934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 10 19:58:54.593934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 10 19:58:54.593934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 10 19:58:54.593934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 10 19:58:54.593934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 10 19:58:54.593934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 10 19:58:54.593934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [parameters: {'user_id': '65a16ae92abc46e19c9cb787cd1cc905', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 10 19:58:54.593934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 10 19:58:54.593934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:54.593934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:54.593934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:54.593934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:54.593934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:54.593934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:54.593934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:54.595819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:54.595819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:54.595819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:54.595819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:54.595819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:54.595819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:54.595819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:54.595819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 10 19:58:54.595819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 10 19:58:54.595819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:54.595819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:54.595819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:54.595819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:54.595819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:54.595819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:54.595819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 10 19:58:54.595819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 10 19:58:54.597672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 10 19:58:54.597672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 10 19:58:54.597672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 10 19:58:54.597672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 10 19:58:54.597672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 10 19:58:54.597672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:54.599897 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 483/963] 10.222.0.65 () {60 vars in 1131 bytes} [Tue May 10 19:58:54 2022] POST /identity/v3/users => generated 160 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:54.601119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46029e27-a6db-4d58-8a27-df5f71f27afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:54.602111 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46029e27-a6db-4d58-8a27-df5f71f27afc None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:54.602675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46029e27-a6db-4d58-8a27-df5f71f27afc None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:54.603152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46029e27-a6db-4d58-8a27-df5f71f27afc None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:54.604089 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46029e27-a6db-4d58-8a27-df5f71f27afc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:54.604678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46029e27-a6db-4d58-8a27-df5f71f27afc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:54.606264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fb245-7d9e-4e0c-87f3-b45a3e2866e0 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:54.607518 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46029e27-a6db-4d58-8a27-df5f71f27afc None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:54.619541 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 481/964] 10.222.0.65 () {56 vars in 1122 bytes} [Tue May 10 19:58:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:54.620416 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46029e27-a6db-4d58-8a27-df5f71f27afc None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:54.723884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46029e27-a6db-4d58-8a27-df5f71f27afc None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:54.731373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fb245-7d9e-4e0c-87f3-b45a3e2866e0 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:54.758884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1f0e6-f7ca-4113-9bcd-2444b9f516a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:54.759686 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1f0e6-f7ca-4113-9bcd-2444b9f516a4 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:54.760065 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1f0e6-f7ca-4113-9bcd-2444b9f516a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:54.760434 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1f0e6-f7ca-4113-9bcd-2444b9f516a4 None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:54.761458 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1f0e6-f7ca-4113-9bcd-2444b9f516a4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:54.761652 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1f0e6-f7ca-4113-9bcd-2444b9f516a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:54.763142 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1f0e6-f7ca-4113-9bcd-2444b9f516a4 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:54.768547 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 482/965] 10.222.0.65 () {56 vars in 1122 bytes} [Tue May 10 19:58:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:54.773621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1f0e6-f7ca-4113-9bcd-2444b9f516a4 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:54.775355 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8a90b-45d7-4759-a701-2af19c4a9a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:54.776247 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8a90b-45d7-4759-a701-2af19c4a9a38 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:54.776622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8a90b-45d7-4759-a701-2af19c4a9a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:54.776988 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8a90b-45d7-4759-a701-2af19c4a9a38 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:54.785426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8a90b-45d7-4759-a701-2af19c4a9a38 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:54.787042 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8a90b-45d7-4759-a701-2af19c4a9a38 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:54.801832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-2ef8a90b-45d7-4759-a701-2af19c4a9a38 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 10 19:58:54.801832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:54.801832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:54.801832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:54.801832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:54.801832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:54.801832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:54.801832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:54.801832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:54.801832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:54.801832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:54.801832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:54.801832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:54.801832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:54.801832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:54.801832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:54.803003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:54.803003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:54.803003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:54.803003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:54.803003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:54.803003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:54.803003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:54.803003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:54.803003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:54.803003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:54.803003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:54.803003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:54.803003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:54.803003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:54.803003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:54.803003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:54.803003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:54.803003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:54.804171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:54.804171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:54.804171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:54.804171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:54.804171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:54.804171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:54.804171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:54.804171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:54.804171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 10 19:58:54.804171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:54.810784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 484/966] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:58:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:54.811571 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8a90b-45d7-4759-a701-2af19c4a9a38 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:54.907440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1f0e6-f7ca-4113-9bcd-2444b9f516a4 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:54.922637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8a90b-45d7-4759-a701-2af19c4a9a38 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:54.941356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c926388-e62d-4cbd-ac82-94e56d40eee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:54.942172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c926388-e62d-4cbd-ac82-94e56d40eee8 None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:54.942551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c926388-e62d-4cbd-ac82-94e56d40eee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:54.942939 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c926388-e62d-4cbd-ac82-94e56d40eee8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:54.943820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c926388-e62d-4cbd-ac82-94e56d40eee8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:54.945241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c926388-e62d-4cbd-ac82-94e56d40eee8 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:54.957587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d5a15-0882-44bd-bea3-aecdcebe4c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:54.958298 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d5a15-0882-44bd-bea3-aecdcebe4c7f None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:54.958585 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d5a15-0882-44bd-bea3-aecdcebe4c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:54.958862 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d5a15-0882-44bd-bea3-aecdcebe4c7f None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:54.959669 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d5a15-0882-44bd-bea3-aecdcebe4c7f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:54.959977 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d5a15-0882-44bd-bea3-aecdcebe4c7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:54.961378 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d5a15-0882-44bd-bea3-aecdcebe4c7f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:55.000120 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 485/967] 10.222.0.65 () {56 vars in 1101 bytes} [Tue May 10 19:58:54 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:55.024485 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 483/968] 10.222.0.65 () {60 vars in 1144 bytes} [Tue May 10 19:58:54 2022] POST /identity/v3/endpoints => generated 349 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:58:55.795646 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4662e9c-c2ce-488b-88cb-75d319d027e8 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:55.796174 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4662e9c-c2ce-488b-88cb-75d319d027e8 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:55.796285 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4662e9c-c2ce-488b-88cb-75d319d027e8 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:55.797789 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 486/969] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:55.802905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e5011-138f-487d-90da-f09f1901e218 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:55.803366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e5011-138f-487d-90da-f09f1901e218 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:55.803571 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e5011-138f-487d-90da-f09f1901e218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:55.821795 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-117e5011-138f-487d-90da-f09f1901e218 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:55.823048 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-117e5011-138f-487d-90da-f09f1901e218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:55.895485 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 484/970] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:55.917502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4662e9c-c2ce-488b-88cb-75d319d027e8 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:56.002017 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ade6c4-6d04-4fd5-b33e-5607f787209f None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:56.002381 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ade6c4-6d04-4fd5-b33e-5607f787209f None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:56.002612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ade6c4-6d04-4fd5-b33e-5607f787209f None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:56.004139 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 485/971] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:58:56.014269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69fb11-835d-430e-92bd-a6757cd20c04 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:56.014557 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69fb11-835d-430e-92bd-a6757cd20c04 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:56.014835 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69fb11-835d-430e-92bd-a6757cd20c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:56.029237 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4662e9c-c2ce-488b-88cb-75d319d027e8 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:56.031191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-4b69fb11-835d-430e-92bd-a6757cd20c04 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:56.032381 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4b69fb11-835d-430e-92bd-a6757cd20c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:56.063047 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16be97-260a-44f5-b06f-0c40601cebae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:56.063757 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16be97-260a-44f5-b06f-0c40601cebae None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:56.064041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16be97-260a-44f5-b06f-0c40601cebae None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:56.064310 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16be97-260a-44f5-b06f-0c40601cebae None admin] PATH_INFO: `/v3/services/compute` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:56.064956 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16be97-260a-44f5-b06f-0c40601cebae None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:56.066332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16be97-260a-44f5-b06f-0c40601cebae None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:56.077770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-4d16be97-260a-44f5-b06f-0c40601cebae None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 10 19:58:56.077770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:56.077770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:56.077770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:56.077770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:56.077770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:56.077770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:56.077770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:56.077770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 10 19:58:56.077770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:56.077770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:56.077770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:56.077770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:56.077770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:56.077770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:56.077770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:56.077770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:56.079134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:56.079134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:56.079134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:56.079134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:56.079134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:56.079134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:56.079134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:56.079134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:56.079134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:56.079134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:56.079134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:56.079134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:56.079134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:56.079134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:56.079134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:56.079134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:56.080173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:56.080173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:56.080173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:56.080173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:56.080173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:56.080173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:56.080173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:56.080173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:56.080173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:56.080173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:56.080173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:56.080173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:56.080173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 10 19:58:56.080173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:56.080173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 487/972] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:56.084755 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16be97-260a-44f5-b06f-0c40601cebae None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:56.101584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 486/973] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:58:56.125407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69fb11-835d-430e-92bd-a6757cd20c04 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:56.184232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16be97-260a-44f5-b06f-0c40601cebae None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:56.221496 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69fb11-835d-430e-92bd-a6757cd20c04 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:56.221748 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022db087-06cd-494f-b07f-711c34110603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:56.222201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022db087-06cd-494f-b07f-711c34110603 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:56.222491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022db087-06cd-494f-b07f-711c34110603 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:56.222766 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022db087-06cd-494f-b07f-711c34110603 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:56.223481 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022db087-06cd-494f-b07f-711c34110603 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:56.223792 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022db087-06cd-494f-b07f-711c34110603 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:56.225159 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022db087-06cd-494f-b07f-711c34110603 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:56.229886 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 488/974] 10.222.0.65 () {56 vars in 1112 bytes} [Tue May 10 19:58:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:56.236005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022db087-06cd-494f-b07f-711c34110603 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:56.257118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ac1b2-5b09-4c36-ab0b-4ccd4c96ef31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:56.257947 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ac1b2-5b09-4c36-ab0b-4ccd4c96ef31 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:56.262984 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ac1b2-5b09-4c36-ab0b-4ccd4c96ef31 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:56.262984 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ac1b2-5b09-4c36-ab0b-4ccd4c96ef31 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:56.271493 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ac1b2-5b09-4c36-ab0b-4ccd4c96ef31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:58:56.279855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-757ac1b2-5b09-4c36-ab0b-4ccd4c96ef31 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 19:58:56.279855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:56.279855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:56.279855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:56.279855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:56.279855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:56.279855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:56.279855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:56.279855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:56.279855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:56.279855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:56.279855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:56.279855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:58:56.279855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:58:56.279855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:58:56.279855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:58:56.281087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:58:56.281087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:58:56.281087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:58:56.281087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:58:56.281087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:56.281087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:56.281087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:56.281087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:56.281087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:56.281087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:56.281087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:56.281087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:56.281087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:56.281087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:56.281087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:56.281087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:56.282060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:56.282060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:56.282060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:56.282060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:56.282060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:58:56.282060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:58:56.282060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:58:56.282060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:58:56.282060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:58:56.282060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:58:56.282060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 19:58:56.282060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:56.282060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 487/975] 10.222.0.65 () {56 vars in 1120 bytes} [Tue May 10 19:58:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:56.287011 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ac1b2-5b09-4c36-ab0b-4ccd4c96ef31 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:56.340499 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022db087-06cd-494f-b07f-711c34110603 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:56.376302 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a394f-452d-4f14-a14f-9ec35b029aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:56.377441 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a394f-452d-4f14-a14f-9ec35b029aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:56.378152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a394f-452d-4f14-a14f-9ec35b029aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:56.378652 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a394f-452d-4f14-a14f-9ec35b029aa8 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:56.379972 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a394f-452d-4f14-a14f-9ec35b029aa8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:56.380501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a394f-452d-4f14-a14f-9ec35b029aa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:56.382783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a394f-452d-4f14-a14f-9ec35b029aa8 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:56.387718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ac1b2-5b09-4c36-ab0b-4ccd4c96ef31 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:56.392655 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 489/976] 10.222.0.65 () {56 vars in 1112 bytes} [Tue May 10 19:58:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:56.393683 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a394f-452d-4f14-a14f-9ec35b029aa8 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:56.428186 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2116d3-73a6-4250-9b4e-c9845d9e7010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:56.428852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2116d3-73a6-4250-9b4e-c9845d9e7010 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:56.429137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2116d3-73a6-4250-9b4e-c9845d9e7010 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:56.429412 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2116d3-73a6-4250-9b4e-c9845d9e7010 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:56.430208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2116d3-73a6-4250-9b4e-c9845d9e7010 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:56.430497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2116d3-73a6-4250-9b4e-c9845d9e7010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:56.431938 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2116d3-73a6-4250-9b4e-c9845d9e7010 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:56.440050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 488/977] 10.222.0.65 () {56 vars in 1115 bytes} [Tue May 10 19:58:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:56.445295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2116d3-73a6-4250-9b4e-c9845d9e7010 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:56.502296 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a394f-452d-4f14-a14f-9ec35b029aa8 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:56.534504 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6b26e-91f3-4af4-b4e3-a00be943d9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:56.535458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6b26e-91f3-4af4-b4e3-a00be943d9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:56.535938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6b26e-91f3-4af4-b4e3-a00be943d9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:56.536414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6b26e-91f3-4af4-b4e3-a00be943d9a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:56.537401 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6b26e-91f3-4af4-b4e3-a00be943d9a0 None admin] RBAC: Adding query filter params (interface=public, service_id=dee2ac26235547a0ae1940faa5dab02d, region_id=RegionOne) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:56.537905 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6b26e-91f3-4af4-b4e3-a00be943d9a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:56.539509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6b26e-91f3-4af4-b4e3-a00be943d9a0 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:56.542243 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2116d3-73a6-4250-9b4e-c9845d9e7010 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:56.544550 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 490/978] 10.222.0.65 () {56 vars in 1251 bytes} [Tue May 10 19:58:56 2022] GET /identity/v3/endpoints?service_id=dee2ac26235547a0ae1940faa5dab02d&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:56.550439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6b26e-91f3-4af4-b4e3-a00be943d9a0 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:56.575772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e2093-30a4-459c-89d7-2ada360fb988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:56.576577 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e2093-30a4-459c-89d7-2ada360fb988 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:56.576834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e2093-30a4-459c-89d7-2ada360fb988 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:56.577105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e2093-30a4-459c-89d7-2ada360fb988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:56.588438 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e2093-30a4-459c-89d7-2ada360fb988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:56.590325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e2093-30a4-459c-89d7-2ada360fb988 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:56.607718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-674e2093-30a4-459c-89d7-2ada360fb988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:56.607718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:56.607718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:56.607718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:56.607718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:56.607718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:56.607718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:56.607718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:56.607718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:56.607718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:56.607718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:56.607718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:56.607718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:56.607718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:56.607718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:56.608846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:56.608846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:56.608846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:56.608846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:56.608846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:56.608846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:56.608846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:56.608846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:56.608846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:56.608846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:56.608846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:56.608846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:56.608846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:56.608846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:56.608846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:56.608846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:56.608846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:56.610878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:56.610878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:56.610878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:56.610878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:56.610878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 489/979] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:56.615712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e2093-30a4-459c-89d7-2ada360fb988 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:56.654177 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6b26e-91f3-4af4-b4e3-a00be943d9a0 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:56.694928 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae82ac9-346e-4616-a951-7124cd859cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:56.695907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae82ac9-346e-4616-a951-7124cd859cbb None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:56.696379 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae82ac9-346e-4616-a951-7124cd859cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:56.696716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae82ac9-346e-4616-a951-7124cd859cbb None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:56.697405 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae82ac9-346e-4616-a951-7124cd859cbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:56.698968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae82ac9-346e-4616-a951-7124cd859cbb None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:56.714446 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e2093-30a4-459c-89d7-2ada360fb988 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:56.719049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 491/980] 10.222.0.65 () {56 vars in 1087 bytes} [Tue May 10 19:58:56 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 19:58:56.751552 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3989e0-ebd2-4a69-a118-186135d94a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:56.752232 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3989e0-ebd2-4a69-a118-186135d94a41 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:56.752509 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3989e0-ebd2-4a69-a118-186135d94a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:56.752784 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3989e0-ebd2-4a69-a118-186135d94a41 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:56.753533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3989e0-ebd2-4a69-a118-186135d94a41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:56.753843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3989e0-ebd2-4a69-a118-186135d94a41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:56.755273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3989e0-ebd2-4a69-a118-186135d94a41 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:56.773339 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 490/981] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:56.775341 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae82ac9-346e-4616-a951-7124cd859cbb None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:56.879080 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae82ac9-346e-4616-a951-7124cd859cbb None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:56.914033 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0fe00-8859-4dea-9662-ac948b0e6ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:56.914857 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0fe00-8859-4dea-9662-ac948b0e6ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:56.915143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0fe00-8859-4dea-9662-ac948b0e6ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:56.915413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0fe00-8859-4dea-9662-ac948b0e6ce1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:56.923103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0fe00-8859-4dea-9662-ac948b0e6ce1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:56.924537 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0fe00-8859-4dea-9662-ac948b0e6ce1 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:56.939148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-e0e0fe00-8859-4dea-9662-ac948b0e6ce1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 19:58:56.939148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:56.939148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:56.939148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:56.939148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:56.939148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:56.939148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:56.939148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:56.939148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:56.939148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:56.939148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:56.939148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:56.939148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:58:56.939148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:58:56.939148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:58:56.939148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:58:56.940193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:56.940193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:56.940193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:58:56.940193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:56.940193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:58:56.940193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:58:56.940193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:56.940193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:56.940193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:56.940193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:56.940193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:56.940193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:56.940193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:56.940193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:56.940193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:56.940193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:56.940193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:56.940193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:56.941294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:56.941294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:56.941294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:58:56.941294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:58:56.941294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:58:56.941294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:58:56.941294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:58:56.941294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:58:56.941294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 19:58:56.941294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:56.941294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 492/982] 10.222.0.65 () {56 vars in 1117 bytes} [Tue May 10 19:58:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:56.948015 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3989e0-ebd2-4a69-a118-186135d94a41 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:57.060609 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3989e0-ebd2-4a69-a118-186135d94a41 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:57.095423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef37cc-2aa9-4054-9295-d8441802d4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:57.096114 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef37cc-2aa9-4054-9295-d8441802d4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:57.096371 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef37cc-2aa9-4054-9295-d8441802d4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:57.096652 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef37cc-2aa9-4054-9295-d8441802d4e1 None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:57.097498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef37cc-2aa9-4054-9295-d8441802d4e1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:57.097803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef37cc-2aa9-4054-9295-d8441802d4e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:57.099278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef37cc-2aa9-4054-9295-d8441802d4e1 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:57.116839 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 491/983] 10.222.0.65 () {56 vars in 1149 bytes} [Tue May 10 19:58:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:57.122687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0fe00-8859-4dea-9662-ac948b0e6ce1 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:57.224144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0fe00-8859-4dea-9662-ac948b0e6ce1 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:57.256976 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c772e7-1f02-408f-af36-9253b077f038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:57.257575 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c772e7-1f02-408f-af36-9253b077f038 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:57.257861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c772e7-1f02-408f-af36-9253b077f038 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:57.258166 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c772e7-1f02-408f-af36-9253b077f038 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:57.320038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb86cf-5d6e-46a9-92f7-a7b2891a706b None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:57.320324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb86cf-5d6e-46a9-92f7-a7b2891a706b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:57.320610 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb86cf-5d6e-46a9-92f7-a7b2891a706b None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:57.322151 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 492/984] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:58:57.326034 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c772e7-1f02-408f-af36-9253b077f038 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:57.326827 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d72c5d-a7da-4fb3-aa30-db4679b5b1be None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:57.327116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d72c5d-a7da-4fb3-aa30-db4679b5b1be None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:57.327396 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d72c5d-a7da-4fb3-aa30-db4679b5b1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:57.327554 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c772e7-1f02-408f-af36-9253b077f038 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:57.343816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-88d72c5d-a7da-4fb3-aa30-db4679b5b1be None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:57.344994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-88d72c5d-a7da-4fb3-aa30-db4679b5b1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:57.427196 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 493/985] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:58:57.450955 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 493/986] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:58:57 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 19:58:57.452334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d72c5d-a7da-4fb3-aa30-db4679b5b1be None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:57.452799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c772e7-1f02-408f-af36-9253b077f038 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:57.548450 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d72c5d-a7da-4fb3-aa30-db4679b5b1be None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:57.549830 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c772e7-1f02-408f-af36-9253b077f038 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:57.583966 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9765a-be3c-45ab-a5f5-b5d0026ed36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:57.584204 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f3c49-f0db-4aab-a390-a179bd8a2baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:57.584769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9765a-be3c-45ab-a5f5-b5d0026ed36e None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:57.584938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f3c49-f0db-4aab-a390-a179bd8a2baa None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:57.585090 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9765a-be3c-45ab-a5f5-b5d0026ed36e None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:57.585238 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f3c49-f0db-4aab-a390-a179bd8a2baa None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:57.585328 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9765a-be3c-45ab-a5f5-b5d0026ed36e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:57.585481 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f3c49-f0db-4aab-a390-a179bd8a2baa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:57.586028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea9765a-be3c-45ab-a5f5-b5d0026ed36e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:57.587615 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea9765a-be3c-45ab-a5f5-b5d0026ed36e None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:57.595717 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f3c49-f0db-4aab-a390-a179bd8a2baa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:57.597137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f3c49-f0db-4aab-a390-a179bd8a2baa None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:57.600820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-aea9765a-be3c-45ab-a5f5-b5d0026ed36e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 19:58:57.600820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:57.600820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:57.600820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:57.600820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:57.600820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:57.600820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:57.600820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:57.600820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 19:58:57.600820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:57.600820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:57.600820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:57.600820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:57.600820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:57.600820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:57.600820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:57.600820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:57.601824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:57.601824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:57.601824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:57.601824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:57.601824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:57.601824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:57.601824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:57.601824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:57.601824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:57.601824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:57.601824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:57.601824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:57.601824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:57.601824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:57.601824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:57.601824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:57.602717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:57.602717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:57.602717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:57.602717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:57.602717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:57.602717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:57.602717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:57.602717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:57.602717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:57.602717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:57.602717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:57.602717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:57.602717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 19:58:57.602717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:57.606952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 494/987] 10.222.0.65 () {56 vars in 1126 bytes} [Tue May 10 19:58:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:57.607870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9765a-be3c-45ab-a5f5-b5d0026ed36e None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:57.615810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-859f3c49-f0db-4aab-a390-a179bd8a2baa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:57.615810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:57.615810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:57.615810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:57.615810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:57.615810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:57.615810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:57.615810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:57.615810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:57.615810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:57.615810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:57.615810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:57.615810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:58:57.615810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:58:57.615810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:58:57.616710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:58:57.616710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:57.616710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:57.616710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:57.616710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:57.616710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:57.616710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:57.616710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:57.616710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:57.616710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:57.616710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:57.616710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:57.616710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:57.616710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:57.616710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:57.616710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:57.616710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:57.617618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:58:57.617618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:58:57.617618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:58:57.617618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:57.617618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 494/988] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:58:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:57.622356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f3c49-f0db-4aab-a390-a179bd8a2baa None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:57.718455 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f3c49-f0db-4aab-a390-a179bd8a2baa None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:57.720673 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9765a-be3c-45ab-a5f5-b5d0026ed36e None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:57.755454 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c540913-a75b-4560-808a-f5953f9a3619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:57.756106 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c540913-a75b-4560-808a-f5953f9a3619 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:57.756388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c540913-a75b-4560-808a-f5953f9a3619 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:57.756657 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c540913-a75b-4560-808a-f5953f9a3619 None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:57.757362 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c540913-a75b-4560-808a-f5953f9a3619 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:57.757666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c540913-a75b-4560-808a-f5953f9a3619 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:57.759185 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c540913-a75b-4560-808a-f5953f9a3619 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:57.760057 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee8fe96-e694-4da5-8abd-17faf0418e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:57.760712 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8fe96-e694-4da5-8abd-17faf0418e0a None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:57.760997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8fe96-e694-4da5-8abd-17faf0418e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:57.761274 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8fe96-e694-4da5-8abd-17faf0418e0a None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:57.761994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee8fe96-e694-4da5-8abd-17faf0418e0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:57.762333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee8fe96-e694-4da5-8abd-17faf0418e0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:57.763679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee8fe96-e694-4da5-8abd-17faf0418e0a None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:57.764436 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 495/989] 10.222.0.65 () {56 vars in 1122 bytes} [Tue May 10 19:58:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:57.769998 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c540913-a75b-4560-808a-f5953f9a3619 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:57.782122 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 495/990] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:57.783279 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee8fe96-e694-4da5-8abd-17faf0418e0a None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:57.869719 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c540913-a75b-4560-808a-f5953f9a3619 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:57.881194 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee8fe96-e694-4da5-8abd-17faf0418e0a None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:57.902725 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced992f3-c300-4e48-b91c-d13eb3094d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:57.903390 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced992f3-c300-4e48-b91c-d13eb3094d4d None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:57.903948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced992f3-c300-4e48-b91c-d13eb3094d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:57.903948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced992f3-c300-4e48-b91c-d13eb3094d4d None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:57.904639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced992f3-c300-4e48-b91c-d13eb3094d4d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:57.904951 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced992f3-c300-4e48-b91c-d13eb3094d4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:57.906869 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced992f3-c300-4e48-b91c-d13eb3094d4d None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:57.911646 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 496/991] 10.222.0.65 () {56 vars in 1122 bytes} [Tue May 10 19:58:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:57.915000 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f384b46-8e40-45e6-8e48-d0b6d28d800a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:57.915798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f384b46-8e40-45e6-8e48-d0b6d28d800a None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:57.916460 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f384b46-8e40-45e6-8e48-d0b6d28d800a None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:57.916758 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f384b46-8e40-45e6-8e48-d0b6d28d800a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:57.917225 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced992f3-c300-4e48-b91c-d13eb3094d4d None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:57.924288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f384b46-8e40-45e6-8e48-d0b6d28d800a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:57.925743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f384b46-8e40-45e6-8e48-d0b6d28d800a None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:57.940427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-9f384b46-8e40-45e6-8e48-d0b6d28d800a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:57.940427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:57.940427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:57.940427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:57.940427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:57.940427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:57.940427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:57.940427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:57.940427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:57.940427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:57.940427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:57.940427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:57.940427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:58:57.940427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:58:57.940427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:58:57.941364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:58:57.941364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:57.941364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:57.941364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:57.941364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:57.941364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:57.941364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:57.941364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:57.941364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:58:57.941364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:57.941364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:57.941364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:57.941364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:57.941364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:57.941364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:57.941364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:57.941364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:57.946917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:58:57.946917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:58:57.946917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:58:57.946917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:58:57.946917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:58:57.946917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:58:57.946917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:58:57.946917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:58:57.946917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 496/992] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:58:57.947418 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f384b46-8e40-45e6-8e48-d0b6d28d800a None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:58.020960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced992f3-c300-4e48-b91c-d13eb3094d4d None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:58.049599 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f384b46-8e40-45e6-8e48-d0b6d28d800a None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:58.054681 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175c957-1e68-4dd3-9d69-6c61db6dd27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:58.055338 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175c957-1e68-4dd3-9d69-6c61db6dd27a None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:58.055626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175c957-1e68-4dd3-9d69-6c61db6dd27a None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:58.055905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175c957-1e68-4dd3-9d69-6c61db6dd27a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:58.056713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175c957-1e68-4dd3-9d69-6c61db6dd27a None admin] RBAC: Adding query filter params (interface=admin, service_id=973d17a2b54144c7a75c5f89f5189a5c, region_id=RegionOne) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:58.057027 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175c957-1e68-4dd3-9d69-6c61db6dd27a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:58.058420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175c957-1e68-4dd3-9d69-6c61db6dd27a None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:58.063441 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 497/993] 10.222.0.65 () {56 vars in 1249 bytes} [Tue May 10 19:58:57 2022] GET /identity/v3/endpoints?service_id=973d17a2b54144c7a75c5f89f5189a5c&interface=admin®ion_id=RegionOne => generated 193 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:58.068888 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175c957-1e68-4dd3-9d69-6c61db6dd27a None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:58.089178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4e30d-9f5c-4a0d-ae57-68acb837d921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:58.090045 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4e30d-9f5c-4a0d-ae57-68acb837d921 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:58.090532 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4e30d-9f5c-4a0d-ae57-68acb837d921 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:58.090948 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4e30d-9f5c-4a0d-ae57-68acb837d921 None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:58.091911 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4e30d-9f5c-4a0d-ae57-68acb837d921 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:58.092222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4e30d-9f5c-4a0d-ae57-68acb837d921 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:58.094209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4e30d-9f5c-4a0d-ae57-68acb837d921 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:58.106265 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 497/994] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:58:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:58.112719 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4e30d-9f5c-4a0d-ae57-68acb837d921 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:58.165358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175c957-1e68-4dd3-9d69-6c61db6dd27a None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:58.198858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42a4d7-4550-4850-9e18-7b6a43d6c3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:58.199516 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42a4d7-4550-4850-9e18-7b6a43d6c3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:58.199803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42a4d7-4550-4850-9e18-7b6a43d6c3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:58.200165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42a4d7-4550-4850-9e18-7b6a43d6c3c9 None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:58.200855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42a4d7-4550-4850-9e18-7b6a43d6c3c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:58.202274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42a4d7-4550-4850-9e18-7b6a43d6c3c9 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:58.207973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4e30d-9f5c-4a0d-ae57-68acb837d921 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:58.208899 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 498/995] 10.222.0.65 () {56 vars in 1087 bytes} [Tue May 10 19:58:58 2022] GET /identity/v3/services => generated 1727 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 19:58:58.243909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43086e-4c8a-4bc1-ab8b-2f95de51095a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:58.244581 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43086e-4c8a-4bc1-ab8b-2f95de51095a None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:58.244868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43086e-4c8a-4bc1-ab8b-2f95de51095a None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:58.245387 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43086e-4c8a-4bc1-ab8b-2f95de51095a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:58.246326 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43086e-4c8a-4bc1-ab8b-2f95de51095a None admin] RBAC: Adding query filter params (role.id=2cb62fd2d3a747ee8827a5d19d6f9f2e, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=7ec2605fd1b84e0ea134b943ed33a605) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:58.246647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43086e-4c8a-4bc1-ab8b-2f95de51095a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:58.248113 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43086e-4c8a-4bc1-ab8b-2f95de51095a None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:58.262387 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 498/996] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:58:58 2022] GET /identity/v3/role_assignments?user.id=7ec2605fd1b84e0ea134b943ed33a605&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=2cb62fd2d3a747ee8827a5d19d6f9f2e => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:58:58.949780 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e99917-8c7b-4fd4-9811-2eecfeb932c8 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:58.950184 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e99917-8c7b-4fd4-9811-2eecfeb932c8 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:58.950392 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e99917-8c7b-4fd4-9811-2eecfeb932c8 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:58.951769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 499/997] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:58:58.957422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152f13d-551d-4bc0-8d3e-f654b0e7464b None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:58.957885 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152f13d-551d-4bc0-8d3e-f654b0e7464b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:58.957965 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152f13d-551d-4bc0-8d3e-f654b0e7464b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:58.974854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-8152f13d-551d-4bc0-8d3e-f654b0e7464b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:58:58.976361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8152f13d-551d-4bc0-8d3e-f654b0e7464b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:58:59.040328 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 499/998] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:58:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:58:59.063290 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e99917-8c7b-4fd4-9811-2eecfeb932c8 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:59.159574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e99917-8c7b-4fd4-9811-2eecfeb932c8 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:59.191510 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a4bf7-ab0e-405e-b8b4-d3463f938209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:59.192294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a4bf7-ab0e-405e-b8b4-d3463f938209 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:59.192577 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a4bf7-ab0e-405e-b8b4-d3463f938209 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:59.192853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a4bf7-ab0e-405e-b8b4-d3463f938209 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:59.193508 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a4bf7-ab0e-405e-b8b4-d3463f938209 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:59.194893 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a4bf7-ab0e-405e-b8b4-d3463f938209 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:59.206309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-8d5a4bf7-ab0e-405e-b8b4-d3463f938209 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 10 19:58:59.206309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:59.206309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:58:59.206309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:58:59.206309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:58:59.206309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:58:59.206309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:58:59.206309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:59.206309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 10 19:58:59.206309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:59.206309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:58:59.206309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:59.206309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:58:59.206309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:58:59.206309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:58:59.206309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:58:59.206309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:58:59.207282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:58:59.207282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:58:59.207282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:58:59.207282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:58:59.207282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:58:59.207282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:58:59.207282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:58:59.207282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:58:59.207282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:58:59.207282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:58:59.207282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:58:59.207282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:58:59.207282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:58:59.207282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:58:59.207282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:58:59.207282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:58:59.208208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:58:59.208208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:58:59.208208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:58:59.208208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:58:59.208208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:58:59.208208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:58:59.208208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:58:59.208208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:58:59.208208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:58:59.208208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:58:59.208208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:58:59.208208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:58:59.208208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 10 19:58:59.208208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:58:59.210441 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 500/999] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:58:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:58:59.213295 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152f13d-551d-4bc0-8d3e-f654b0e7464b None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:59.307395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152f13d-551d-4bc0-8d3e-f654b0e7464b None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:59.342265 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f27a4-e442-4ed4-983b-ca592af301e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:59.343798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f27a4-e442-4ed4-983b-ca592af301e9 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:59.344248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f27a4-e442-4ed4-983b-ca592af301e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:59.344685 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f27a4-e442-4ed4-983b-ca592af301e9 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:59.345548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f27a4-e442-4ed4-983b-ca592af301e9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:59.346003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f27a4-e442-4ed4-983b-ca592af301e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:59.347561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f27a4-e442-4ed4-983b-ca592af301e9 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:59.352854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 500/1000] 10.222.0.65 () {56 vars in 1112 bytes} [Tue May 10 19:58:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:59.358070 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a4bf7-ab0e-405e-b8b4-d3463f938209 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:59.471571 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a4bf7-ab0e-405e-b8b4-d3463f938209 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:59.504505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86923fb6-4a85-4bf8-ad13-550289a2c630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:59.505495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86923fb6-4a85-4bf8-ad13-550289a2c630 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:59.506014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86923fb6-4a85-4bf8-ad13-550289a2c630 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:59.506536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86923fb6-4a85-4bf8-ad13-550289a2c630 None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:59.507461 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86923fb6-4a85-4bf8-ad13-550289a2c630 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:58:59.507982 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86923fb6-4a85-4bf8-ad13-550289a2c630 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:59.509537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86923fb6-4a85-4bf8-ad13-550289a2c630 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:59.519949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 501/1001] 10.222.0.65 () {56 vars in 1112 bytes} [Tue May 10 19:58:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:58:59.520759 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f27a4-e442-4ed4-983b-ca592af301e9 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:58:59.620139 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f27a4-e442-4ed4-983b-ca592af301e9 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:58:59.653144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53238c93-46c6-4946-861d-4e1e18c122fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:58:59.653980 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53238c93-46c6-4946-861d-4e1e18c122fc None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:58:59.654461 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53238c93-46c6-4946-861d-4e1e18c122fc None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:58:59.654880 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53238c93-46c6-4946-861d-4e1e18c122fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:58:59.656301 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53238c93-46c6-4946-861d-4e1e18c122fc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:58:59.657798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53238c93-46c6-4946-861d-4e1e18c122fc None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:58:59.759941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 501/1002] 10.222.0.65 () {60 vars in 1144 bytes} [Tue May 10 19:58:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 19:59:00.441021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e623122-c6ef-45dc-a517-379565f1f063 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:00.441466 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e623122-c6ef-45dc-a517-379565f1f063 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:00.441647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e623122-c6ef-45dc-a517-379565f1f063 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:00.443111 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 502/1003] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:00.448297 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f27a68-7cc3-4713-8b61-10fa2fba4507 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:00.448717 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f27a68-7cc3-4713-8b61-10fa2fba4507 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:00.448911 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f27a68-7cc3-4713-8b61-10fa2fba4507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:00.465565 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-d4f27a68-7cc3-4713-8b61-10fa2fba4507 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:00.466765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-d4f27a68-7cc3-4713-8b61-10fa2fba4507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:00.524599 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1e38a-fe7d-4628-bbbe-1b808918f64a None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:00.524929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1e38a-fe7d-4628-bbbe-1b808918f64a None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:00.525165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1e38a-fe7d-4628-bbbe-1b808918f64a None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:00.526680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 503/1004] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:59:00.532080 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658901c8-9be6-420a-b695-30b8b40130fd None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:00.532364 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658901c8-9be6-420a-b695-30b8b40130fd None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:00.532639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658901c8-9be6-420a-b695-30b8b40130fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:00.538131 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 502/1005] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:00 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:59:00.549188 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-658901c8-9be6-420a-b695-30b8b40130fd None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:00.550380 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-658901c8-9be6-420a-b695-30b8b40130fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:00.558983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f27a68-7cc3-4713-8b61-10fa2fba4507 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:00.617158 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 504/1006] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:00 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:59:00.638748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658901c8-9be6-420a-b695-30b8b40130fd None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:00.661139 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f27a68-7cc3-4713-8b61-10fa2fba4507 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:00.694564 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e26e8-d9f1-4640-8213-3ed950cc0e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:00.695370 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e26e8-d9f1-4640-8213-3ed950cc0e0a None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:00.695650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e26e8-d9f1-4640-8213-3ed950cc0e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:00.695939 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e26e8-d9f1-4640-8213-3ed950cc0e0a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:00.696582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e26e8-d9f1-4640-8213-3ed950cc0e0a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:00.697948 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e26e8-d9f1-4640-8213-3ed950cc0e0a None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:00.710402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-540e26e8-d9f1-4640-8213-3ed950cc0e0a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 19:59:00.710402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:00.710402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:59:00.710402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:59:00.710402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:59:00.710402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:59:00.710402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:59:00.710402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:59:00.710402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 19:59:00.710402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:59:00.710402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:59:00.710402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:59:00.710402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:00.710402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:00.710402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:00.710402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:00.710402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:00.711423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:00.711423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:00.711423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:00.711423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:00.711423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:00.711423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:00.711423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:59:00.711423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:59:00.711423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:59:00.711423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:59:00.711423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:00.711423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:00.711423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:00.711423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:00.711423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:00.711423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:00.717324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:00.717324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:59:00.717324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:00.717324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:00.717324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:00.717324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:00.717324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:00.717324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:00.717324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:00.717324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:00.717324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:59:00.717324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:59:00.717324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 19:59:00.717324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:59:00.717324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 503/1007] 10.222.0.65 () {56 vars in 1126 bytes} [Tue May 10 19:59:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:59:00.718746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e26e8-d9f1-4640-8213-3ed950cc0e0a None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:00.737694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658901c8-9be6-420a-b695-30b8b40130fd None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:00.771677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808e092-539b-4c7c-be88-ca7095c987b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:00.772453 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808e092-539b-4c7c-be88-ca7095c987b2 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:00.772734 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808e092-539b-4c7c-be88-ca7095c987b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:00.773009 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808e092-539b-4c7c-be88-ca7095c987b2 None admin] PATH_INFO: `/v3/services/image` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:00.773657 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808e092-539b-4c7c-be88-ca7095c987b2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:00.775103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808e092-539b-4c7c-be88-ca7095c987b2 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:00.786737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-d808e092-539b-4c7c-be88-ca7095c987b2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 10 19:59:00.786737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:00.786737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:59:00.786737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:59:00.786737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:59:00.786737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:59:00.786737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:59:00.786737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:59:00.786737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 10 19:59:00.786737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:00.786737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:59:00.786737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:00.786737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:00.786737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:00.786737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:00.786737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:00.786737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:00.787759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:00.787759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:00.787759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:00.787759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:00.787759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:00.787759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:00.787759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:59:00.787759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:59:00.787759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:59:00.787759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:59:00.787759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:00.787759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:00.787759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:00.787759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:00.787759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:00.787759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:00.788601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:00.788601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:59:00.788601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:00.788601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:00.788601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:00.788601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:00.788601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:00.788601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:00.788601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:00.788601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:00.788601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:59:00.788601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:59:00.788601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 10 19:59:00.788601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:00.792982 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 505/1008] 10.222.0.65 () {56 vars in 1105 bytes} [Tue May 10 19:59:00 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:59:00.794053 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808e092-539b-4c7c-be88-ca7095c987b2 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:00.814431 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e26e8-d9f1-4640-8213-3ed950cc0e0a None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:00.867189 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c8bde-e99d-41d0-aeff-86d1a6aabb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:00.868292 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c8bde-e99d-41d0-aeff-86d1a6aabb9d None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:00.868783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c8bde-e99d-41d0-aeff-86d1a6aabb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:00.869293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c8bde-e99d-41d0-aeff-86d1a6aabb9d None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:00.870603 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c8bde-e99d-41d0-aeff-86d1a6aabb9d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:00.871144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c8bde-e99d-41d0-aeff-86d1a6aabb9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:00.873653 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c8bde-e99d-41d0-aeff-86d1a6aabb9d None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:00.885684 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 504/1009] 10.222.0.65 () {56 vars in 1122 bytes} [Tue May 10 19:59:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:00.887849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c8bde-e99d-41d0-aeff-86d1a6aabb9d None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:00.895194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808e092-539b-4c7c-be88-ca7095c987b2 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:00.939012 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e94db-da75-4a7b-9edd-6d2014026cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:00.940107 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e94db-da75-4a7b-9edd-6d2014026cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:00.940600 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e94db-da75-4a7b-9edd-6d2014026cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:00.941079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e94db-da75-4a7b-9edd-6d2014026cf8 None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:00.942406 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e94db-da75-4a7b-9edd-6d2014026cf8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:00.942916 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e94db-da75-4a7b-9edd-6d2014026cf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:00.945427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e94db-da75-4a7b-9edd-6d2014026cf8 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:00.958312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 506/1010] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:59:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:00.959570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e94db-da75-4a7b-9edd-6d2014026cf8 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:00.992246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c8bde-e99d-41d0-aeff-86d1a6aabb9d None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:01.024782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa4400-6e11-4be8-b2d4-962a52c24706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:01.025525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa4400-6e11-4be8-b2d4-962a52c24706 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:01.025873 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa4400-6e11-4be8-b2d4-962a52c24706 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:01.026240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa4400-6e11-4be8-b2d4-962a52c24706 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:01.026949 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa4400-6e11-4be8-b2d4-962a52c24706 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:01.027944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa4400-6e11-4be8-b2d4-962a52c24706 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:01.028588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa4400-6e11-4be8-b2d4-962a52c24706 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:01.038342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 505/1011] 10.222.0.65 () {56 vars in 1122 bytes} [Tue May 10 19:59:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:01.039398 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa4400-6e11-4be8-b2d4-962a52c24706 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:01.057691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e94db-da75-4a7b-9edd-6d2014026cf8 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:01.095072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e81502-2885-49be-9e53-bdba52d561ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:01.095748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e81502-2885-49be-9e53-bdba52d561ad None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:01.097744 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e81502-2885-49be-9e53-bdba52d561ad None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:01.097744 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e81502-2885-49be-9e53-bdba52d561ad None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:01.097744 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e81502-2885-49be-9e53-bdba52d561ad None admin] RBAC: Adding query filter params (type=image) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:01.097744 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e81502-2885-49be-9e53-bdba52d561ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:01.098719 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e81502-2885-49be-9e53-bdba52d561ad None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:01.118114 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 507/1012] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:59:00 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:01.155687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa4400-6e11-4be8-b2d4-962a52c24706 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:01.190945 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a2fa3-3381-44ff-bcdf-d0103b439f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:01.191614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a2fa3-3381-44ff-bcdf-d0103b439f51 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:01.191899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a2fa3-3381-44ff-bcdf-d0103b439f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:01.192174 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a2fa3-3381-44ff-bcdf-d0103b439f51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:01.192936 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a2fa3-3381-44ff-bcdf-d0103b439f51 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:01.194355 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a2fa3-3381-44ff-bcdf-d0103b439f51 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:01.279368 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 506/1013] 10.222.0.65 () {60 vars in 1144 bytes} [Tue May 10 19:59:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:59:02.163484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13df34-2db9-47ac-b0cf-c2e1ce7e77b8 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:02.164002 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13df34-2db9-47ac-b0cf-c2e1ce7e77b8 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:02.164002 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13df34-2db9-47ac-b0cf-c2e1ce7e77b8 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:02.165363 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 508/1014] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:02.170729 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d0678-2a24-4797-99ae-0f44dc2d3795 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:02.171156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d0678-2a24-4797-99ae-0f44dc2d3795 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:02.171359 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d0678-2a24-4797-99ae-0f44dc2d3795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:02.187694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-9a1d0678-2a24-4797-99ae-0f44dc2d3795 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:02.188852 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-9a1d0678-2a24-4797-99ae-0f44dc2d3795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:02.258754 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 507/1015] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:02 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:59:02.283195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13df34-2db9-47ac-b0cf-c2e1ce7e77b8 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:02.381371 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13df34-2db9-47ac-b0cf-c2e1ce7e77b8 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:02.414620 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef13017-4761-407c-ae1a-c34f6dbfb1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:02.418388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef13017-4761-407c-ae1a-c34f6dbfb1be None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:02.418388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef13017-4761-407c-ae1a-c34f6dbfb1be None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:02.418388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef13017-4761-407c-ae1a-c34f6dbfb1be None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:02.431201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef13017-4761-407c-ae1a-c34f6dbfb1be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:59:02.439668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-fef13017-4761-407c-ae1a-c34f6dbfb1be None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 19:59:02.439668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:02.439668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:02.439668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:02.439668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:02.439668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:02.439668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:02.439668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:02.439668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:02.439668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:02.439668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:02.439668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:02.439668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:59:02.439668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:59:02.439668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:59:02.439668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:59:02.440584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:59:02.440584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:59:02.440584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:59:02.440584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:59:02.440584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:02.440584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:02.440584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:02.440584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:02.440584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:02.440584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:02.440584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:02.440584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:59:02.440584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:02.440584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:02.440584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:02.440584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:02.441462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:02.441462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:02.441462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:02.441462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:02.441462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:59:02.441462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:59:02.441462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:59:02.441462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:59:02.441462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:59:02.441462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:59:02.441462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 19:59:02.441462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:02.445748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 509/1016] 10.222.0.65 () {56 vars in 1120 bytes} [Tue May 10 19:59:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:59:02.447236 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d0678-2a24-4797-99ae-0f44dc2d3795 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:02.540114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d0678-2a24-4797-99ae-0f44dc2d3795 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:02.571963 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1234c-bfea-4e51-a9dd-9dfbe3400236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:02.572616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1234c-bfea-4e51-a9dd-9dfbe3400236 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:02.572894 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1234c-bfea-4e51-a9dd-9dfbe3400236 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:02.573162 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1234c-bfea-4e51-a9dd-9dfbe3400236 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:02.573902 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1234c-bfea-4e51-a9dd-9dfbe3400236 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:02.574241 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1234c-bfea-4e51-a9dd-9dfbe3400236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:02.575593 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1234c-bfea-4e51-a9dd-9dfbe3400236 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:02.583971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 508/1017] 10.222.0.65 () {56 vars in 1115 bytes} [Tue May 10 19:59:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:02.589466 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef13017-4761-407c-ae1a-c34f6dbfb1be None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:02.683179 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef13017-4761-407c-ae1a-c34f6dbfb1be None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:02.716079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1eb75d-02f4-4675-9c89-a72088600bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:02.716860 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1eb75d-02f4-4675-9c89-a72088600bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:02.717140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1eb75d-02f4-4675-9c89-a72088600bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:02.717412 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1eb75d-02f4-4675-9c89-a72088600bd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:02.727060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1eb75d-02f4-4675-9c89-a72088600bd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:02.728482 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1eb75d-02f4-4675-9c89-a72088600bd7 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:02.747234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-bb1eb75d-02f4-4675-9c89-a72088600bd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:59:02.747234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:02.747234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:02.747234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:02.747234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:02.747234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:02.747234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:02.747234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:02.747234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:02.747234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:02.747234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:02.747234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:02.747234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:59:02.747234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:59:02.747234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:59:02.748152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:59:02.748152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:02.748152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:02.748152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:02.748152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:02.748152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:02.748152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:02.748152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:02.748152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:59:02.748152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:02.748152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:02.748152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:02.748152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:02.748152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:02.748152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:02.748152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:02.748152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:02.754325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:59:02.754325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:59:02.754325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:59:02.754325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:02.754325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 510/1018] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:59:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:59:02.756120 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1234c-bfea-4e51-a9dd-9dfbe3400236 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:02.851356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1234c-bfea-4e51-a9dd-9dfbe3400236 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:02.884260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81dd46-9784-410e-8cf9-2813d0149171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:02.884921 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81dd46-9784-410e-8cf9-2813d0149171 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:02.885197 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81dd46-9784-410e-8cf9-2813d0149171 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:02.885465 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81dd46-9784-410e-8cf9-2813d0149171 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:02.886212 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81dd46-9784-410e-8cf9-2813d0149171 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:02.886514 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81dd46-9784-410e-8cf9-2813d0149171 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:02.887864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81dd46-9784-410e-8cf9-2813d0149171 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:02.899655 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 509/1019] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:02.905169 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1eb75d-02f4-4675-9c89-a72088600bd7 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:03.002645 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1eb75d-02f4-4675-9c89-a72088600bd7 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:03.035313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd3376-9d99-4a78-909a-a0679c475d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:03.036105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd3376-9d99-4a78-909a-a0679c475d51 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:03.036391 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd3376-9d99-4a78-909a-a0679c475d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:03.036669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd3376-9d99-4a78-909a-a0679c475d51 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:03.043836 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd3376-9d99-4a78-909a-a0679c475d51 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:03.045271 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd3376-9d99-4a78-909a-a0679c475d51 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:03.059960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-0ebd3376-9d99-4a78-909a-a0679c475d51 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 19:59:03.059960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:03.059960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:03.059960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:03.059960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:03.059960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:03.059960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:03.059960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:03.059960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:03.059960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:03.059960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:03.059960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:03.059960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:59:03.059960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:59:03.059960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:59:03.059960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:59:03.060892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:03.060892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:03.060892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:59:03.060892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:59:03.060892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:59:03.060892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:59:03.060892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:03.060892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:03.060892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:03.060892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:03.060892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:03.060892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:59:03.060892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:03.060892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:03.060892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:03.060892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:03.060892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:03.060892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:03.067081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:03.067081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:03.067081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:59:03.067081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:59:03.067081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:59:03.067081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:59:03.067081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:59:03.067081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:59:03.067081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 19:59:03.067081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:03.067081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 511/1020] 10.222.0.65 () {56 vars in 1093 bytes} [Tue May 10 19:59:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:59:03.069237 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81dd46-9784-410e-8cf9-2813d0149171 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:03.166325 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81dd46-9784-410e-8cf9-2813d0149171 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:03.203326 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19b0a0-b0ae-4cc4-aa8c-50e6cfd012a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:03.204096 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19b0a0-b0ae-4cc4-aa8c-50e6cfd012a9 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:03.204426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19b0a0-b0ae-4cc4-aa8c-50e6cfd012a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:03.204692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19b0a0-b0ae-4cc4-aa8c-50e6cfd012a9 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:03.205589 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19b0a0-b0ae-4cc4-aa8c-50e6cfd012a9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:03.205902 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19b0a0-b0ae-4cc4-aa8c-50e6cfd012a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:03.207426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19b0a0-b0ae-4cc4-aa8c-50e6cfd012a9 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:03.230334 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 510/1021] 10.222.0.65 () {56 vars in 1133 bytes} [Tue May 10 19:59:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:03.232692 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd3376-9d99-4a78-909a-a0679c475d51 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:03.333425 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd3376-9d99-4a78-909a-a0679c475d51 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:03.370060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391d2ac-7da1-4ea0-b875-441f8a6a323a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:03.370752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391d2ac-7da1-4ea0-b875-441f8a6a323a None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:03.371039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391d2ac-7da1-4ea0-b875-441f8a6a323a None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:03.371319 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391d2ac-7da1-4ea0-b875-441f8a6a323a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:03.374311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1b673-10b0-444c-9f0f-88372239dd2c None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:03.374611 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1b673-10b0-444c-9f0f-88372239dd2c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:03.374896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1b673-10b0-444c-9f0f-88372239dd2c None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:03.376410 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 511/1022] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:03.381160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469269c-5a8a-4d56-9abd-9b2169af0ec3 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:03.381452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469269c-5a8a-4d56-9abd-9b2169af0ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:03.381733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469269c-5a8a-4d56-9abd-9b2169af0ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:03.397766 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-8469269c-5a8a-4d56-9abd-9b2169af0ec3 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:03.399042 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8469269c-5a8a-4d56-9abd-9b2169af0ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:03.445955 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391d2ac-7da1-4ea0-b875-441f8a6a323a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:03.447494 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391d2ac-7da1-4ea0-b875-441f8a6a323a None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:03.466631 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 512/1023] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:03 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:59:03.494923 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469269c-5a8a-4d56-9abd-9b2169af0ec3 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:03.546530 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 512/1024] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:59:03 2022] GET /identity/v3/auth/tokens => generated 1863 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 10 19:59:03.547923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643c5a6-2847-4590-91f3-4db609fa68c2 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:03.548215 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643c5a6-2847-4590-91f3-4db609fa68c2 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:03.548493 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643c5a6-2847-4590-91f3-4db609fa68c2 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:03.551133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 513/1025] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:03.553661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643c5a6-2847-4590-91f3-4db609fa68c2 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:03.591916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469269c-5a8a-4d56-9abd-9b2169af0ec3 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:03.624117 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ebbab-359c-4471-992c-5869759f7f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:03.624768 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ebbab-359c-4471-992c-5869759f7f06 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:03.625047 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ebbab-359c-4471-992c-5869759f7f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:03.625344 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ebbab-359c-4471-992c-5869759f7f06 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:03.626059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ebbab-359c-4471-992c-5869759f7f06 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:03.627441 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ebbab-359c-4471-992c-5869759f7f06 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:03.649137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643c5a6-2847-4590-91f3-4db609fa68c2 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:03.681511 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d59e0-2eee-4e5a-8dd0-66c77dd40b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:03.682360 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d59e0-2eee-4e5a-8dd0-66c77dd40b8f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:03.682645 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d59e0-2eee-4e5a-8dd0-66c77dd40b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:03.682919 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d59e0-2eee-4e5a-8dd0-66c77dd40b8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:03.692537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d59e0-2eee-4e5a-8dd0-66c77dd40b8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:03.693964 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d59e0-2eee-4e5a-8dd0-66c77dd40b8f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:03.711630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-4f1d59e0-2eee-4e5a-8dd0-66c77dd40b8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:59:03.711630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:03.711630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:03.711630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:03.711630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:03.711630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:03.711630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:03.711630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:03.711630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:03.711630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:03.711630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:03.711630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:03.711630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:59:03.711630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:59:03.711630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:59:03.712677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:59:03.712677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:03.712677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:03.712677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:03.712677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:03.712677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:03.712677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:03.712677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:03.712677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:59:03.712677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:03.712677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:03.712677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:03.712677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:03.712677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:03.712677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:03.712677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:03.712677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:03.713662 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:59:03.713662 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:59:03.713662 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:59:03.713662 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:03.713662 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 514/1026] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:59:03.713953 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b92eb-dcf9-4d97-8180-3f480be2bc85 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:03.714237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b92eb-dcf9-4d97-8180-3f480be2bc85 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:03.714516 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b92eb-dcf9-4d97-8180-3f480be2bc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:03.736236 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 513/1027] 10.222.0.65 () {60 vars in 1141 bytes} [Tue May 10 19:59:03 2022] POST /identity/v3/services => generated 242 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 19:59:03.736236 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ebbab-359c-4471-992c-5869759f7f06 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:03.738051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-030b92eb-dcf9-4d97-8180-3f480be2bc85 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:03.739525 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-030b92eb-dcf9-4d97-8180-3f480be2bc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:03.821071 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 515/1028] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:59:03.844339 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ebbab-359c-4471-992c-5869759f7f06 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:03.845187 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b92eb-dcf9-4d97-8180-3f480be2bc85 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:03.879553 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7abb8-562d-4c74-b7b1-4b684c7a96da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:03.880233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7abb8-562d-4c74-b7b1-4b684c7a96da None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:03.880514 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7abb8-562d-4c74-b7b1-4b684c7a96da None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:03.880794 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7abb8-562d-4c74-b7b1-4b684c7a96da None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:03.881544 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7abb8-562d-4c74-b7b1-4b684c7a96da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:03.881849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7abb8-562d-4c74-b7b1-4b684c7a96da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:03.883290 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7abb8-562d-4c74-b7b1-4b684c7a96da None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:03.896345 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 514/1029] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:03.901887 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7abb8-562d-4c74-b7b1-4b684c7a96da None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:03.956624 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b92eb-dcf9-4d97-8180-3f480be2bc85 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:03.991783 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb6ff4-c37a-4b4c-99aa-4edf93853252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:03.992616 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb6ff4-c37a-4b4c-99aa-4edf93853252 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:03.992900 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb6ff4-c37a-4b4c-99aa-4edf93853252 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:03.993169 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb6ff4-c37a-4b4c-99aa-4edf93853252 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:03.994361 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb6ff4-c37a-4b4c-99aa-4edf93853252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:03.995773 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb6ff4-c37a-4b4c-99aa-4edf93853252 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:03.998110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 516/1030] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:59:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:04.002866 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb6ff4-c37a-4b4c-99aa-4edf93853252 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:04.003210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7abb8-562d-4c74-b7b1-4b684c7a96da None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:04.036103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265526ad-8bfb-4529-8e25-e42ea371d39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:04.036891 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265526ad-8bfb-4529-8e25-e42ea371d39d None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:04.037170 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265526ad-8bfb-4529-8e25-e42ea371d39d None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:04.037442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265526ad-8bfb-4529-8e25-e42ea371d39d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:04.044211 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265526ad-8bfb-4529-8e25-e42ea371d39d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:04.045647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265526ad-8bfb-4529-8e25-e42ea371d39d None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:04.059734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-265526ad-8bfb-4529-8e25-e42ea371d39d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:59:04.059734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:04.059734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:04.059734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:04.059734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:04.059734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:04.059734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:04.059734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:04.059734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:04.059734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:04.059734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:04.059734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:04.059734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:59:04.059734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:59:04.059734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:59:04.060650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:59:04.060650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:04.060650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:04.060650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:04.060650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:04.060650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:04.060650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:04.060650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:04.060650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:59:04.060650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:04.060650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:04.060650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:04.060650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:04.060650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:04.060650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:04.060650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:04.060650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:04.061592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:59:04.061592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:59:04.061592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:59:04.061592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:59:04.061592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:59:04.061592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:59:04.061592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:59:04.061592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:59:04.061592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 515/1031] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:59:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:59:04.066863 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265526ad-8bfb-4529-8e25-e42ea371d39d None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:04.100274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb6ff4-c37a-4b4c-99aa-4edf93853252 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:04.133997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f56bfc4-6402-4736-9ec3-379e673130b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:04.134708 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56bfc4-6402-4736-9ec3-379e673130b6 None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:04.134994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56bfc4-6402-4736-9ec3-379e673130b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:04.135324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56bfc4-6402-4736-9ec3-379e673130b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:04.136052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f56bfc4-6402-4736-9ec3-379e673130b6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:04.137516 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f56bfc4-6402-4736-9ec3-379e673130b6 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:04.164199 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265526ad-8bfb-4529-8e25-e42ea371d39d None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:04.196164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e26cea-95cf-4f96-92b9-2d6320dcb0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:04.196819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e26cea-95cf-4f96-92b9-2d6320dcb0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:04.197103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e26cea-95cf-4f96-92b9-2d6320dcb0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:04.197727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e26cea-95cf-4f96-92b9-2d6320dcb0c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:04.198199 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e26cea-95cf-4f96-92b9-2d6320dcb0c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:04.198508 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e26cea-95cf-4f96-92b9-2d6320dcb0c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:04.199940 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e26cea-95cf-4f96-92b9-2d6320dcb0c5 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:04.212319 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 516/1032] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:59:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:04.218447 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e26cea-95cf-4f96-92b9-2d6320dcb0c5 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:04.274243 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 517/1033] 10.222.0.65 () {60 vars in 1141 bytes} [Tue May 10 19:59:04 2022] POST /identity/v3/projects => generated 311 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 19:59:04.322702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e26cea-95cf-4f96-92b9-2d6320dcb0c5 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:04.360210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128164f-bc05-4f06-90b2-c11c722e20e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:04.361125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128164f-bc05-4f06-90b2-c11c722e20e2 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:04.361569 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128164f-bc05-4f06-90b2-c11c722e20e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:04.362363 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128164f-bc05-4f06-90b2-c11c722e20e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:04.363414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128164f-bc05-4f06-90b2-c11c722e20e2 None admin] RBAC: Adding query filter params (role.id=2cb62fd2d3a747ee8827a5d19d6f9f2e, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=ae922b2da4814d10bed1fa345700ac04) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:04.366483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128164f-bc05-4f06-90b2-c11c722e20e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:04.366483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128164f-bc05-4f06-90b2-c11c722e20e2 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:04.372234 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 517/1034] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:59:04 2022] GET /identity/v3/role_assignments?user.id=ae922b2da4814d10bed1fa345700ac04&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=2cb62fd2d3a747ee8827a5d19d6f9f2e => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:05.982021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f783afb-0240-4587-8858-4d720ca4645a None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:05.982616 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f783afb-0240-4587-8858-4d720ca4645a None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:05.982616 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f783afb-0240-4587-8858-4d720ca4645a None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:05.984659 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 518/1035] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:05.989783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c356ee-6327-4298-a19c-23801c830405 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:05.990275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c356ee-6327-4298-a19c-23801c830405 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:05.990452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c356ee-6327-4298-a19c-23801c830405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:06.006677 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-b8c356ee-6327-4298-a19c-23801c830405 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:06.007864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b8c356ee-6327-4298-a19c-23801c830405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:06.073462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 518/1036] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:59:06.096636 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f783afb-0240-4587-8858-4d720ca4645a None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:06.195448 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f783afb-0240-4587-8858-4d720ca4645a None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:06.228062 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1130e-19a5-44ab-9402-d7aa68270569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:06.228850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1130e-19a5-44ab-9402-d7aa68270569 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:06.229136 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1130e-19a5-44ab-9402-d7aa68270569 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:06.229415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1130e-19a5-44ab-9402-d7aa68270569 None admin] PATH_INFO: `/v3/services/image` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:06.230075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b1130e-19a5-44ab-9402-d7aa68270569 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:06.231463 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b1130e-19a5-44ab-9402-d7aa68270569 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:06.243246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-31b1130e-19a5-44ab-9402-d7aa68270569 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 10 19:59:06.243246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:06.243246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:59:06.243246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:59:06.243246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:59:06.243246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:59:06.243246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:59:06.243246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:59:06.243246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 10 19:59:06.243246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:06.243246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:59:06.243246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:06.243246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:06.243246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:06.243246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:06.243246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:06.243246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:06.244249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:06.244249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:06.244249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:06.244249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:06.244249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:06.244249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:06.244249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:59:06.244249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:59:06.244249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:59:06.244249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:59:06.244249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:06.244249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:06.244249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:06.244249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:06.244249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:06.244249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:06.245128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:06.245128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:59:06.245128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:06.245128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:06.245128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:06.245128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:06.245128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:06.245128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:06.245128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:06.245128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:06.245128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:59:06.245128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:59:06.245128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 10 19:59:06.245128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:06.252998 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 519/1037] 10.222.0.65 () {56 vars in 1105 bytes} [Tue May 10 19:59:06 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:59:06.253128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c356ee-6327-4298-a19c-23801c830405 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:06.349751 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c356ee-6327-4298-a19c-23801c830405 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:06.388374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7b930-25fa-46fd-a891-d7a2b17636a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:06.389463 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7b930-25fa-46fd-a891-d7a2b17636a7 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:06.390567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7b930-25fa-46fd-a891-d7a2b17636a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:06.390856 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7b930-25fa-46fd-a891-d7a2b17636a7 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:06.391620 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7b930-25fa-46fd-a891-d7a2b17636a7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:06.391938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7b930-25fa-46fd-a891-d7a2b17636a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:06.393373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7b930-25fa-46fd-a891-d7a2b17636a7 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:06.403735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 519/1038] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:59:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:06.405006 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1130e-19a5-44ab-9402-d7aa68270569 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:06.507319 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1130e-19a5-44ab-9402-d7aa68270569 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:06.534389 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e75c8c-5ea9-441b-87d2-e8560e6b8711 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:06.534688 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e75c8c-5ea9-441b-87d2-e8560e6b8711 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:06.534954 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e75c8c-5ea9-441b-87d2-e8560e6b8711 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:06.536320 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 520/1039] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:59:06.540625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fa0c1-8f77-4328-98cd-569b19c1d11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:06.541164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695126b4-caef-4460-bae8-99fe26ad6699 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:06.541452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695126b4-caef-4460-bae8-99fe26ad6699 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:06.541732 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695126b4-caef-4460-bae8-99fe26ad6699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:06.542259 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fa0c1-8f77-4328-98cd-569b19c1d11f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:06.542763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fa0c1-8f77-4328-98cd-569b19c1d11f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:06.543249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fa0c1-8f77-4328-98cd-569b19c1d11f None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:06.544346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544fa0c1-8f77-4328-98cd-569b19c1d11f None admin] RBAC: Adding query filter params (type=image) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:06.544858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544fa0c1-8f77-4328-98cd-569b19c1d11f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:06.547022 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544fa0c1-8f77-4328-98cd-569b19c1d11f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:06.552130 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 520/1040] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:59:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:06.557707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fa0c1-8f77-4328-98cd-569b19c1d11f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:06.559929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-695126b4-caef-4460-bae8-99fe26ad6699 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:06.561122 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-695126b4-caef-4460-bae8-99fe26ad6699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:06.626454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 521/1041] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:59:06.627762 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634847f-5298-4340-b3cf-372b1db19e4d None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:06.628112 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634847f-5298-4340-b3cf-372b1db19e4d None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:06.628317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634847f-5298-4340-b3cf-372b1db19e4d None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:06.629691 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 522/1042] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:06.634706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0a041-197f-462d-89d8-8681b5a93c52 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:06.634997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0a041-197f-462d-89d8-8681b5a93c52 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:06.635275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0a041-197f-462d-89d8-8681b5a93c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:06.651300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-5aa0a041-197f-462d-89d8-8681b5a93c52 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:06.652502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-5aa0a041-197f-462d-89d8-8681b5a93c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:06.652910 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fa0c1-8f77-4328-98cd-569b19c1d11f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:06.685273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c887d-ed2c-48f7-8702-56a7f5c59bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:06.685924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c887d-ed2c-48f7-8702-56a7f5c59bde None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:06.686252 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c887d-ed2c-48f7-8702-56a7f5c59bde None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:06.687105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c887d-ed2c-48f7-8702-56a7f5c59bde None admin] PATH_INFO: `/v3/endpoints` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:06.687879 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c887d-ed2c-48f7-8702-56a7f5c59bde None admin] RBAC: Adding query filter params (interface=public, service_id=3a1db6b6c93e4fa78a7a1f785d8340bf, region_id=RegionOne) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:06.688193 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c887d-ed2c-48f7-8702-56a7f5c59bde None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:06.689557 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c887d-ed2c-48f7-8702-56a7f5c59bde None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:06.694484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 521/1043] 10.222.0.65 () {56 vars in 1251 bytes} [Tue May 10 19:59:06 2022] GET /identity/v3/endpoints?service_id=3a1db6b6c93e4fa78a7a1f785d8340bf&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:06.695254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c887d-ed2c-48f7-8702-56a7f5c59bde None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:06.716686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 523/1044] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:59:06.717739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0a041-197f-462d-89d8-8681b5a93c52 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:06.789554 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c887d-ed2c-48f7-8702-56a7f5c59bde None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:06.811106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0a041-197f-462d-89d8-8681b5a93c52 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:06.821662 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37300e-0c0a-458d-a3b4-a5c35846f2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:06.822467 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37300e-0c0a-458d-a3b4-a5c35846f2bc None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:06.822752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37300e-0c0a-458d-a3b4-a5c35846f2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:06.823029 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37300e-0c0a-458d-a3b4-a5c35846f2bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:06.824198 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f37300e-0c0a-458d-a3b4-a5c35846f2bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:06.825621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f37300e-0c0a-458d-a3b4-a5c35846f2bc None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:06.827436 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 522/1045] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:59:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:06.828031 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37300e-0c0a-458d-a3b4-a5c35846f2bc None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:06.844621 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc52b00-786d-4f2f-adf0-a0ed6ccb441c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:06.845282 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc52b00-786d-4f2f-adf0-a0ed6ccb441c None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:06.845567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc52b00-786d-4f2f-adf0-a0ed6ccb441c None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:06.845841 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc52b00-786d-4f2f-adf0-a0ed6ccb441c None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:06.846562 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc52b00-786d-4f2f-adf0-a0ed6ccb441c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:06.847915 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc52b00-786d-4f2f-adf0-a0ed6ccb441c None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:06.854622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 524/1046] 10.222.0.65 () {56 vars in 1087 bytes} [Tue May 10 19:59:06 2022] GET /identity/v3/services => generated 1957 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 19:59:06.855080 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc52b00-786d-4f2f-adf0-a0ed6ccb441c None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:06.929375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37300e-0c0a-458d-a3b4-a5c35846f2bc None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:06.959606 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc52b00-786d-4f2f-adf0-a0ed6ccb441c None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:06.963522 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c75b43-a931-47f0-bd09-07319cc18a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:06.964345 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c75b43-a931-47f0-bd09-07319cc18a15 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:06.964623 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c75b43-a931-47f0-bd09-07319cc18a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:06.964897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c75b43-a931-47f0-bd09-07319cc18a15 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:06.976667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c75b43-a931-47f0-bd09-07319cc18a15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:59:06.985786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-64c75b43-a931-47f0-bd09-07319cc18a15 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 19:59:06.985786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:06.985786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:06.985786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:06.985786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:06.985786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:06.985786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:06.985786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:06.985786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:06.985786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:06.985786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:06.985786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:06.985786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:59:06.985786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:59:06.985786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:59:06.985786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:59:06.992884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:59:06.992884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:59:06.992884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:59:06.992884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:59:06.992884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:06.992884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:06.992884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:06.992884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:06.992884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:06.992884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:06.992884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:06.992884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:59:06.992884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:06.992884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:06.992884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:06.992884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:06.993763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:06.993763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:06.993763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:06.993763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:06.993763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:59:06.993763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:59:06.993763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:59:06.993763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:59:06.993763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:59:06.993763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:59:06.993763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 19:59:06.993763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:06.993763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 523/1047] 10.222.0.65 () {56 vars in 1120 bytes} [Tue May 10 19:59:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:59:06.993763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c75b43-a931-47f0-bd09-07319cc18a15 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:06.998287 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a137bf-b693-443e-88f4-bc8b5e67d05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:06.999014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a137bf-b693-443e-88f4-bc8b5e67d05f None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:06.999304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a137bf-b693-443e-88f4-bc8b5e67d05f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:06.999585 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a137bf-b693-443e-88f4-bc8b5e67d05f None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:07.000314 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a137bf-b693-443e-88f4-bc8b5e67d05f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:07.001754 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a137bf-b693-443e-88f4-bc8b5e67d05f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:07.094405 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c75b43-a931-47f0-bd09-07319cc18a15 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:07.114473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 525/1048] 10.222.0.65 () {60 vars in 1132 bytes} [Tue May 10 19:59:06 2022] POST /identity/v3/users => generated 285 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:59:07.127641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19798d-6db5-4850-ac72-7ee3c59b2f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:07.128588 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19798d-6db5-4850-ac72-7ee3c59b2f65 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:07.129021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19798d-6db5-4850-ac72-7ee3c59b2f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:07.129481 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19798d-6db5-4850-ac72-7ee3c59b2f65 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:07.130450 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19798d-6db5-4850-ac72-7ee3c59b2f65 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:07.130934 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19798d-6db5-4850-ac72-7ee3c59b2f65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:07.132508 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19798d-6db5-4850-ac72-7ee3c59b2f65 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:07.142448 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 524/1049] 10.222.0.65 () {56 vars in 1115 bytes} [Tue May 10 19:59:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:07.148211 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a137bf-b693-443e-88f4-bc8b5e67d05f None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:07.258522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a137bf-b693-443e-88f4-bc8b5e67d05f None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:07.294005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ee906-ec24-4a17-a19a-22392c877910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:07.294833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ee906-ec24-4a17-a19a-22392c877910 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:07.295119 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ee906-ec24-4a17-a19a-22392c877910 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:07.295394 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ee906-ec24-4a17-a19a-22392c877910 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:07.307093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ee906-ec24-4a17-a19a-22392c877910 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:07.308541 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ee906-ec24-4a17-a19a-22392c877910 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:07.326543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-496ee906-ec24-4a17-a19a-22392c877910 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:59:07.326543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:07.326543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:07.326543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:07.326543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:07.326543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:07.326543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:07.326543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:07.326543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:07.326543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:07.326543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:07.326543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:07.326543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:59:07.326543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:59:07.326543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:59:07.328019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:59:07.328019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:07.328019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:07.328019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:07.328019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:07.328019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:07.328019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:07.328019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:07.328019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:59:07.328019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:07.328019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:07.328019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:07.328019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:07.328019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:07.328019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:07.328019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:07.328019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:07.328937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:59:07.328937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:59:07.328937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:59:07.328937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:59:07.328937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 526/1050] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:59:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:59:07.334381 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19798d-6db5-4850-ac72-7ee3c59b2f65 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:07.436269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19798d-6db5-4850-ac72-7ee3c59b2f65 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:07.470484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3aa11-ce4d-44c9-95ff-2e0b55793fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:07.471146 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3aa11-ce4d-44c9-95ff-2e0b55793fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:07.471430 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3aa11-ce4d-44c9-95ff-2e0b55793fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:07.471707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3aa11-ce4d-44c9-95ff-2e0b55793fb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:07.472432 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f3aa11-ce4d-44c9-95ff-2e0b55793fb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:07.472739 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f3aa11-ce4d-44c9-95ff-2e0b55793fb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:07.474115 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f3aa11-ce4d-44c9-95ff-2e0b55793fb1 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:07.486911 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 525/1051] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:07.491827 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ee906-ec24-4a17-a19a-22392c877910 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:07.591297 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ee906-ec24-4a17-a19a-22392c877910 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:07.624487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6dcc7-2361-45b0-827c-8f324e1d162f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:07.625280 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6dcc7-2361-45b0-827c-8f324e1d162f None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:07.625563 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6dcc7-2361-45b0-827c-8f324e1d162f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:07.625837 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6dcc7-2361-45b0-827c-8f324e1d162f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:07.634297 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d6dcc7-2361-45b0-827c-8f324e1d162f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:07.635860 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d6dcc7-2361-45b0-827c-8f324e1d162f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:07.650164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-98d6dcc7-2361-45b0-827c-8f324e1d162f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 19:59:07.650164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:07.650164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:07.650164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:07.650164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:07.650164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:07.650164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:07.650164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:07.650164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:07.650164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:07.650164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:07.650164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:07.650164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:59:07.650164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:59:07.650164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:59:07.650164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:59:07.651137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:07.651137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:07.651137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:59:07.651137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:59:07.651137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:59:07.651137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:59:07.651137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:07.651137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:07.651137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:07.651137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:07.651137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:07.651137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:59:07.651137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:07.651137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:07.651137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:07.651137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:07.651137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:07.651137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:07.652793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:07.652793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:07.652793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:59:07.652793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:59:07.652793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:59:07.652793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:59:07.652793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:59:07.652793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:59:07.652793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 19:59:07.652793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:59:07.653599 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 527/1052] 10.222.0.65 () {56 vars in 1093 bytes} [Tue May 10 19:59:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:59:07.657262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3aa11-ce4d-44c9-95ff-2e0b55793fb1 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:07.755611 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3aa11-ce4d-44c9-95ff-2e0b55793fb1 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:07.787843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ced1e-5df7-4faf-8db0-74a3b4b29822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:07.788503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ced1e-5df7-4faf-8db0-74a3b4b29822 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:07.788786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ced1e-5df7-4faf-8db0-74a3b4b29822 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:07.789057 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ced1e-5df7-4faf-8db0-74a3b4b29822 None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:07.789893 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ced1e-5df7-4faf-8db0-74a3b4b29822 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:07.790230 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ced1e-5df7-4faf-8db0-74a3b4b29822 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:07.791647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ced1e-5df7-4faf-8db0-74a3b4b29822 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:07.809020 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 526/1053] 10.222.0.65 () {56 vars in 1133 bytes} [Tue May 10 19:59:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:07.814140 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6dcc7-2361-45b0-827c-8f324e1d162f None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:07.908828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6dcc7-2361-45b0-827c-8f324e1d162f None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:07.940990 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16129f-cd39-43e2-afb0-3e188a0d3422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:07.941778 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16129f-cd39-43e2-afb0-3e188a0d3422 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:07.942060 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16129f-cd39-43e2-afb0-3e188a0d3422 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:07.942362 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16129f-cd39-43e2-afb0-3e188a0d3422 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:07.953079 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16129f-cd39-43e2-afb0-3e188a0d3422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:07.954566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16129f-cd39-43e2-afb0-3e188a0d3422 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:07.971331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-2d16129f-cd39-43e2-afb0-3e188a0d3422 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:59:07.971331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:07.971331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:07.971331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:07.971331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:07.971331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:07.971331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:07.971331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:07.971331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:07.971331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:07.971331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:07.971331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:07.971331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:59:07.971331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:59:07.971331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:59:07.972260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:59:07.972260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:07.972260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:07.972260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:07.972260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:07.972260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:07.972260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:07.972260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:07.972260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:59:07.972260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:07.972260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:07.972260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:07.972260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:07.972260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:07.972260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:07.972260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:07.972260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:07.973181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:59:07.973181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:59:07.973181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:59:07.973181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:59:07.974043 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 528/1054] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:59:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:59:07.978401 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ced1e-5df7-4faf-8db0-74a3b4b29822 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:08.077543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ced1e-5df7-4faf-8db0-74a3b4b29822 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:08.109940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ebe0e-ebe5-4b8e-9d4e-42d0279bbbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:08.110682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ebe0e-ebe5-4b8e-9d4e-42d0279bbbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:08.110973 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ebe0e-ebe5-4b8e-9d4e-42d0279bbbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:08.111248 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ebe0e-ebe5-4b8e-9d4e-42d0279bbbe7 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:08.112226 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ebe0e-ebe5-4b8e-9d4e-42d0279bbbe7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:08.112539 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ebe0e-ebe5-4b8e-9d4e-42d0279bbbe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:08.113900 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ebe0e-ebe5-4b8e-9d4e-42d0279bbbe7 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:08.129831 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 527/1055] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:08.131308 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16129f-cd39-43e2-afb0-3e188a0d3422 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:08.226192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16129f-cd39-43e2-afb0-3e188a0d3422 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:08.258983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe9d0d-d9ca-4dcd-99b6-6ba8dc617af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:08.265609 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe9d0d-d9ca-4dcd-99b6-6ba8dc617af4 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:08.265609 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe9d0d-d9ca-4dcd-99b6-6ba8dc617af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:08.265609 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe9d0d-d9ca-4dcd-99b6-6ba8dc617af4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:08.266961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe9d0d-d9ca-4dcd-99b6-6ba8dc617af4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:08.268398 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe9d0d-d9ca-4dcd-99b6-6ba8dc617af4 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:08.282548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-cabe9d0d-d9ca-4dcd-99b6-6ba8dc617af4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:59:08.282548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:08.282548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:08.282548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:08.282548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:08.282548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:08.282548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:08.282548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:08.282548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:08.282548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:08.282548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:08.282548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:08.282548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:59:08.282548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:59:08.282548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:59:08.283667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:59:08.283667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:08.283667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:08.283667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:08.283667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:08.283667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:08.283667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:08.283667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:08.283667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:59:08.283667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:08.283667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:08.283667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:08.283667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:08.283667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:08.283667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:08.283667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:08.283667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:08.284587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:59:08.284587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:59:08.284587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:59:08.284587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:59:08.284587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:59:08.284587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:59:08.284587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:59:08.284587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:59:08.286482 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 529/1056] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:59:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:59:08.289693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ebe0e-ebe5-4b8e-9d4e-42d0279bbbe7 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:08.387380 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ebe0e-ebe5-4b8e-9d4e-42d0279bbbe7 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:08.423406 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95056432-e579-42e1-9cf3-6995c36a746f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:08.424073 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95056432-e579-42e1-9cf3-6995c36a746f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:08.424358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95056432-e579-42e1-9cf3-6995c36a746f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:08.424635 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95056432-e579-42e1-9cf3-6995c36a746f None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:08.425924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95056432-e579-42e1-9cf3-6995c36a746f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:08.426241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95056432-e579-42e1-9cf3-6995c36a746f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:08.428062 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95056432-e579-42e1-9cf3-6995c36a746f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:08.441562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 528/1057] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:59:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:08.447578 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe9d0d-d9ca-4dcd-99b6-6ba8dc617af4 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:08.542476 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe9d0d-d9ca-4dcd-99b6-6ba8dc617af4 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:08.575078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd330d63-cc6c-47a7-94f2-c49726496e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:08.575959 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd330d63-cc6c-47a7-94f2-c49726496e58 None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:08.576242 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd330d63-cc6c-47a7-94f2-c49726496e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:08.576514 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd330d63-cc6c-47a7-94f2-c49726496e58 None admin] PATH_INFO: `/v3/projects/a006477d1df94390b599621446bd0f02/users/ae922b2da4814d10bed1fa345700ac04/roles/2cb62fd2d3a747ee8827a5d19d6f9f2e` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:08.578770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd330d63-cc6c-47a7-94f2-c49726496e58 None admin] RBAC: Authorizing `identity:create_grant(project_id=a006477d1df94390b599621446bd0f02, user_id=ae922b2da4814d10bed1fa345700ac04, role_id=2cb62fd2d3a747ee8827a5d19d6f9f2e)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:08.580213 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd330d63-cc6c-47a7-94f2-c49726496e58 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:08.650546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 530/1058] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:59:08 2022] PUT /identity/v3/projects/a006477d1df94390b599621446bd0f02/users/ae922b2da4814d10bed1fa345700ac04/roles/2cb62fd2d3a747ee8827a5d19d6f9f2e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 19:59:09.086473 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cdc02e-3f5a-4397-a41b-d43cfe03fe81 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:09.086994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cdc02e-3f5a-4397-a41b-d43cfe03fe81 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:09.087072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cdc02e-3f5a-4397-a41b-d43cfe03fe81 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:09.088404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 529/1059] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:09.094302 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d6c80-86ba-41ec-b596-ec6c2ee25f8e None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:09.094594 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d6c80-86ba-41ec-b596-ec6c2ee25f8e None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:09.094875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d6c80-86ba-41ec-b596-ec6c2ee25f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:09.111168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-302d6c80-86ba-41ec-b596-ec6c2ee25f8e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:09.112610 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-302d6c80-86ba-41ec-b596-ec6c2ee25f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:09.178054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 531/1060] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:59:09.203874 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cdc02e-3f5a-4397-a41b-d43cfe03fe81 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:09.306010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cdc02e-3f5a-4397-a41b-d43cfe03fe81 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:09.339872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2c13f-b41b-47be-b81a-a823e4b6a415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:09.340579 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2c13f-b41b-47be-b81a-a823e4b6a415 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:09.340915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2c13f-b41b-47be-b81a-a823e4b6a415 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:09.341191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2c13f-b41b-47be-b81a-a823e4b6a415 None admin] PATH_INFO: `/v3/services/image` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:09.341848 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2c13f-b41b-47be-b81a-a823e4b6a415 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:09.343368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2c13f-b41b-47be-b81a-a823e4b6a415 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:09.354830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-94c2c13f-b41b-47be-b81a-a823e4b6a415 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 10 19:59:09.354830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:09.354830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 19:59:09.354830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 19:59:09.354830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 19:59:09.354830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 19:59:09.354830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 19:59:09.354830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:59:09.354830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 10 19:59:09.354830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:09.354830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:59:09.354830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:09.354830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:09.354830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:09.354830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:09.354830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:09.354830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:09.355826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:09.355826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:09.355826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:09.355826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:09.355826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:09.355826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:09.355826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 19:59:09.355826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 19:59:09.355826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 19:59:09.355826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 19:59:09.355826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:09.355826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:09.355826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:09.355826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:09.355826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:09.355826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:09.356707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:09.356707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:59:09.356707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:09.356707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:09.356707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:09.356707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:09.356707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:09.356707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:09.356707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:09.356707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:09.356707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 19:59:09.356707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 19:59:09.356707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 10 19:59:09.356707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:09.358134 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 530/1061] 10.222.0.65 () {56 vars in 1105 bytes} [Tue May 10 19:59:09 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:59:09.362387 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d6c80-86ba-41ec-b596-ec6c2ee25f8e None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:09.422675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7a133-a56b-4bba-afa8-ce003308c5b2 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:09.422967 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7a133-a56b-4bba-afa8-ce003308c5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:09.423247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7a133-a56b-4bba-afa8-ce003308c5b2 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:09.426455 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 531/1062] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:59:09.431308 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41715274-1c9f-4749-a0d6-745b590270a3 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:09.431783 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41715274-1c9f-4749-a0d6-745b590270a3 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:09.432220 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41715274-1c9f-4749-a0d6-745b590270a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:09.449406 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-41715274-1c9f-4749-a0d6-745b590270a3 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:09.450836 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-41715274-1c9f-4749-a0d6-745b590270a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:09.466480 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d6c80-86ba-41ec-b596-ec6c2ee25f8e None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:09.501678 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5c6fa-9026-405a-bad1-794514aa0018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:09.502384 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5c6fa-9026-405a-bad1-794514aa0018 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:09.502669 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5c6fa-9026-405a-bad1-794514aa0018 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:09.502949 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5c6fa-9026-405a-bad1-794514aa0018 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:09.503671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5c6fa-9026-405a-bad1-794514aa0018 None admin] RBAC: Adding query filter params (name=image) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:09.503975 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5c6fa-9026-405a-bad1-794514aa0018 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:09.505341 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5c6fa-9026-405a-bad1-794514aa0018 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:09.510273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 532/1063] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:59:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:09.516398 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5c6fa-9026-405a-bad1-794514aa0018 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:09.532997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 532/1064] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:59:09.556560 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41715274-1c9f-4749-a0d6-745b590270a3 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:09.622619 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5c6fa-9026-405a-bad1-794514aa0018 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:09.653554 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41715274-1c9f-4749-a0d6-745b590270a3 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:09.655261 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ccda6-32b7-4d1a-9cfe-0918152d0dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:09.655914 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ccda6-32b7-4d1a-9cfe-0918152d0dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:09.656199 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ccda6-32b7-4d1a-9cfe-0918152d0dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:09.656475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ccda6-32b7-4d1a-9cfe-0918152d0dd9 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:09.657184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ccda6-32b7-4d1a-9cfe-0918152d0dd9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:09.657492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ccda6-32b7-4d1a-9cfe-0918152d0dd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:09.658882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ccda6-32b7-4d1a-9cfe-0918152d0dd9 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:09.663538 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 533/1065] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:59:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:09.669143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ccda6-32b7-4d1a-9cfe-0918152d0dd9 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:09.689673 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d5b14-3894-477b-ac53-65e0729fa039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:09.690503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d5b14-3894-477b-ac53-65e0729fa039 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:09.690784 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d5b14-3894-477b-ac53-65e0729fa039 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:09.691054 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d5b14-3894-477b-ac53-65e0729fa039 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:09.701608 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d5b14-3894-477b-ac53-65e0729fa039 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:59:09.709171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-9f6d5b14-3894-477b-ac53-65e0729fa039 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:59:09.709171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:09.709171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:09.709171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:09.709171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:09.709171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:09.709171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:09.709171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:09.709171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:09.709171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:09.709171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:09.709171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:09.709171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:59:09.709171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:59:09.709171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:59:09.709171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:59:09.710210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:59:09.710210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:59:09.710210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:59:09.710210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:59:09.710210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:09.710210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:09.710210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:09.710210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:09.710210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:09.710210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:09.710210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:09.710210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:59:09.710210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:09.710210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:09.710210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:09.710210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:09.711082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:09.711082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:09.711082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:09.711082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:09.711082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:59:09.711082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:59:09.711082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:59:09.711082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:59:09.711082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:59:09.711082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:59:09.711082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:59:09.711082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:09.711082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 533/1066] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:59:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:59:09.716137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d5b14-3894-477b-ac53-65e0729fa039 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:09.767545 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ccda6-32b7-4d1a-9cfe-0918152d0dd9 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:09.802171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd7a0f-2ed5-404d-b55b-76dccfdd176c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:09.802823 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd7a0f-2ed5-404d-b55b-76dccfdd176c None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:09.803102 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd7a0f-2ed5-404d-b55b-76dccfdd176c None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:09.803375 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd7a0f-2ed5-404d-b55b-76dccfdd176c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:09.804111 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd7a0f-2ed5-404d-b55b-76dccfdd176c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:09.805477 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd7a0f-2ed5-404d-b55b-76dccfdd176c None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:09.819164 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d5b14-3894-477b-ac53-65e0729fa039 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:09.856774 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521d1a3-81c8-49a8-97e4-3a8d5d371928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:09.857442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521d1a3-81c8-49a8-97e4-3a8d5d371928 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:09.857723 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521d1a3-81c8-49a8-97e4-3a8d5d371928 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:09.857998 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521d1a3-81c8-49a8-97e4-3a8d5d371928 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:09.859336 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521d1a3-81c8-49a8-97e4-3a8d5d371928 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:09.859645 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521d1a3-81c8-49a8-97e4-3a8d5d371928 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:09.861013 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521d1a3-81c8-49a8-97e4-3a8d5d371928 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:09.869475 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 534/1067] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:59:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:09.875057 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521d1a3-81c8-49a8-97e4-3a8d5d371928 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:09.925637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 534/1068] 10.222.0.65 () {60 vars in 1144 bytes} [Tue May 10 19:59:09 2022] POST /identity/v3/endpoints => generated 327 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:59:09.975609 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521d1a3-81c8-49a8-97e4-3a8d5d371928 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:10.017917 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4a1b5-bc65-4075-a9a1-328d36fd41b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:10.018835 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4a1b5-bc65-4075-a9a1-328d36fd41b3 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:10.019123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4a1b5-bc65-4075-a9a1-328d36fd41b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:10.019401 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4a1b5-bc65-4075-a9a1-328d36fd41b3 None admin] PATH_INFO: `/v3/users/b6dc9c6540fe48b6939c76e4368ecc4a` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:10.034946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4a1b5-bc65-4075-a9a1-328d36fd41b3 None admin] RBAC: Authorizing `identity:get_user(user_id=b6dc9c6540fe48b6939c76e4368ecc4a)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:10.036412 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4a1b5-bc65-4075-a9a1-328d36fd41b3 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:10.038393 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 535/1069] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:59:09 2022] GET /identity/v3/users/b6dc9c6540fe48b6939c76e4368ecc4a => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:10.044078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd7a0f-2ed5-404d-b55b-76dccfdd176c None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:10.150766 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd7a0f-2ed5-404d-b55b-76dccfdd176c None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:10.183807 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded8605-450b-4ec6-ab84-99c86029e9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:10.184458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded8605-450b-4ec6-ab84-99c86029e9bf None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:10.184730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded8605-450b-4ec6-ab84-99c86029e9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:10.185005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded8605-450b-4ec6-ab84-99c86029e9bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:10.249378 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded8605-450b-4ec6-ab84-99c86029e9bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:10.250862 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded8605-450b-4ec6-ab84-99c86029e9bf None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:10.346520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 535/1070] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:59:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 10 19:59:10.352072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4a1b5-bc65-4075-a9a1-328d36fd41b3 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:10.451647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4a1b5-bc65-4075-a9a1-328d36fd41b3 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:10.486197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c57e4-22ee-4c09-8e1b-6ebefe6ba14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:10.487309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c57e4-22ee-4c09-8e1b-6ebefe6ba14d None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:10.487590 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c57e4-22ee-4c09-8e1b-6ebefe6ba14d None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:10.487862 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c57e4-22ee-4c09-8e1b-6ebefe6ba14d None admin] PATH_INFO: `/v3/projects/58e0c54a4af644aba7536199faf19434` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:10.489088 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c57e4-22ee-4c09-8e1b-6ebefe6ba14d None admin] RBAC: Authorizing `identity:get_project(project_id=58e0c54a4af644aba7536199faf19434)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:10.490536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c57e4-22ee-4c09-8e1b-6ebefe6ba14d None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:10.497114 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 536/1071] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:59:10 2022] GET /identity/v3/projects/58e0c54a4af644aba7536199faf19434 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:10.498393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded8605-450b-4ec6-ab84-99c86029e9bf None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:10.593162 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded8605-450b-4ec6-ab84-99c86029e9bf None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:10.625835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a780940-3846-460f-90b8-8e0c1a1d969b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:10.626509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a780940-3846-460f-90b8-8e0c1a1d969b None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:10.626788 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a780940-3846-460f-90b8-8e0c1a1d969b None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:10.627056 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a780940-3846-460f-90b8-8e0c1a1d969b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:10.627885 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a780940-3846-460f-90b8-8e0c1a1d969b None admin] RBAC: Adding query filter params (role.id=5c87e804d0764309ba08ad6e06b23a87, scope.project.id=58e0c54a4af644aba7536199faf19434, user.id=b6dc9c6540fe48b6939c76e4368ecc4a) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:10.628191 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a780940-3846-460f-90b8-8e0c1a1d969b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:10.629598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a780940-3846-460f-90b8-8e0c1a1d969b None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:10.642167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 536/1072] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:59:10 2022] GET /identity/v3/role_assignments?user.id=b6dc9c6540fe48b6939c76e4368ecc4a&scope.project.id=58e0c54a4af644aba7536199faf19434&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:10.885719 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03411696-ea4b-4994-80ac-c23910a681ce None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:10.886117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03411696-ea4b-4994-80ac-c23910a681ce None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:10.886311 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03411696-ea4b-4994-80ac-c23910a681ce None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:10.887713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 537/1073] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:10.893355 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4fbdd-32fd-4155-b2a7-11acef950b74 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:10.893654 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4fbdd-32fd-4155-b2a7-11acef950b74 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:10.893934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4fbdd-32fd-4155-b2a7-11acef950b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:10.910853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-68c4fbdd-32fd-4155-b2a7-11acef950b74 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:10.912028 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-68c4fbdd-32fd-4155-b2a7-11acef950b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:10.978145 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 537/1074] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:59:11.000477 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03411696-ea4b-4994-80ac-c23910a681ce None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:11.100521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03411696-ea4b-4994-80ac-c23910a681ce None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:11.135829 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cccb6-a45f-4d3c-b7e3-6467d5d6def8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:11.136557 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cccb6-a45f-4d3c-b7e3-6467d5d6def8 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:11.136837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cccb6-a45f-4d3c-b7e3-6467d5d6def8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:11.137109 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cccb6-a45f-4d3c-b7e3-6467d5d6def8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:11.151726 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81cccb6-a45f-4d3c-b7e3-6467d5d6def8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:59:11.160487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-b81cccb6-a45f-4d3c-b7e3-6467d5d6def8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 19:59:11.160487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:11.160487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:11.160487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:11.160487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:11.160487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:11.160487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:11.160487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:11.160487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:11.160487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:11.160487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:11.160487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:11.160487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:59:11.160487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:59:11.160487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:59:11.160487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:59:11.161451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:59:11.161451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:59:11.161451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:59:11.161451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:59:11.161451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:11.161451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:11.161451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:11.161451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:11.161451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:11.161451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:11.161451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:11.161451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:59:11.161451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:11.161451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:11.161451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:11.161451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:11.162365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:11.162365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:11.162365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:11.162365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:11.162365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:59:11.162365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:59:11.162365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:59:11.162365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:59:11.162365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:59:11.162365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:59:11.162365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 19:59:11.162365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:11.166399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 538/1075] 10.222.0.65 () {56 vars in 1120 bytes} [Tue May 10 19:59:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:59:11.167720 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4fbdd-32fd-4155-b2a7-11acef950b74 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:11.261350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4fbdd-32fd-4155-b2a7-11acef950b74 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:11.294587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe35afe-fbc4-4582-85f8-988e6eeb4ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:11.295251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe35afe-fbc4-4582-85f8-988e6eeb4ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:11.295560 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe35afe-fbc4-4582-85f8-988e6eeb4ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:11.295830 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe35afe-fbc4-4582-85f8-988e6eeb4ab6 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:11.296578 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe35afe-fbc4-4582-85f8-988e6eeb4ab6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:11.296882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe35afe-fbc4-4582-85f8-988e6eeb4ab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:11.298264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe35afe-fbc4-4582-85f8-988e6eeb4ab6 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:11.310677 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 538/1076] 10.222.0.65 () {56 vars in 1115 bytes} [Tue May 10 19:59:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:11.312254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cccb6-a45f-4d3c-b7e3-6467d5d6def8 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:11.409197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cccb6-a45f-4d3c-b7e3-6467d5d6def8 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:11.442618 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986a02b-2230-4fa5-90a4-04aaec3738d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:11.443459 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986a02b-2230-4fa5-90a4-04aaec3738d7 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:11.443749 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986a02b-2230-4fa5-90a4-04aaec3738d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:11.444029 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986a02b-2230-4fa5-90a4-04aaec3738d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:11.455667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986a02b-2230-4fa5-90a4-04aaec3738d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:11.457108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986a02b-2230-4fa5-90a4-04aaec3738d7 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:11.473647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-6986a02b-2230-4fa5-90a4-04aaec3738d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:59:11.473647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:11.473647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:11.473647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:11.473647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:11.473647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:11.473647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:11.473647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:11.473647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:11.473647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:11.473647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:11.473647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:11.473647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:59:11.473647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:59:11.473647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:59:11.474798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:59:11.474798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:11.474798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:11.474798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:11.474798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:11.474798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:11.474798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:11.474798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:11.474798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:59:11.474798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:11.474798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:11.474798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:11.474798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:11.474798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:11.474798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:11.474798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:11.474798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:11.480469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:59:11.480469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:59:11.480469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:59:11.480469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:11.480469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 539/1077] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:59:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:59:11.482497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe35afe-fbc4-4582-85f8-988e6eeb4ab6 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:11.590073 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe35afe-fbc4-4582-85f8-988e6eeb4ab6 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:11.623262 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfeea9b-6d17-474c-b631-f31cba261f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:11.624555 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfeea9b-6d17-474c-b631-f31cba261f0d None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:11.624836 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfeea9b-6d17-474c-b631-f31cba261f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:11.625105 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfeea9b-6d17-474c-b631-f31cba261f0d None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:11.625827 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfeea9b-6d17-474c-b631-f31cba261f0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:11.626156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfeea9b-6d17-474c-b631-f31cba261f0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:11.627516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfeea9b-6d17-474c-b631-f31cba261f0d None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:11.643989 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 539/1078] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:11.645191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986a02b-2230-4fa5-90a4-04aaec3738d7 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:11.744512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986a02b-2230-4fa5-90a4-04aaec3738d7 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:11.777769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4686a-9991-44de-a5e7-0ec3bbb35899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:11.778631 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4686a-9991-44de-a5e7-0ec3bbb35899 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:11.778918 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4686a-9991-44de-a5e7-0ec3bbb35899 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:11.779195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4686a-9991-44de-a5e7-0ec3bbb35899 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:11.786706 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4686a-9991-44de-a5e7-0ec3bbb35899 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:11.788146 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4686a-9991-44de-a5e7-0ec3bbb35899 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:11.802713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-b0f4686a-9991-44de-a5e7-0ec3bbb35899 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 19:59:11.802713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:11.802713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:11.802713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:11.802713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:11.802713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:11.802713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:11.802713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:11.802713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:11.802713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:11.802713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:11.802713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:11.802713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 19:59:11.802713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 19:59:11.802713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 19:59:11.802713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 19:59:11.803680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:11.803680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:11.803680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 19:59:11.803680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:59:11.803680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 19:59:11.803680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 19:59:11.803680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:11.803680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:11.803680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:11.803680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:11.803680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:11.803680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:59:11.803680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:11.803680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:11.803680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:11.803680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:11.803680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:11.803680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:11.804628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:11.804628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:11.804628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 19:59:11.804628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 19:59:11.804628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 19:59:11.804628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 19:59:11.804628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 19:59:11.804628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 19:59:11.804628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 19:59:11.804628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:11.804628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 540/1079] 10.222.0.65 () {56 vars in 1093 bytes} [Tue May 10 19:59:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:59:11.809729 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfeea9b-6d17-474c-b631-f31cba261f0d None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:11.921158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfeea9b-6d17-474c-b631-f31cba261f0d None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:11.953461 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c62a97-739e-43c8-90a9-95dad3ae3edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:11.954144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c62a97-739e-43c8-90a9-95dad3ae3edd None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:11.954429 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c62a97-739e-43c8-90a9-95dad3ae3edd None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:11.954700 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c62a97-739e-43c8-90a9-95dad3ae3edd None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:11.955535 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c62a97-739e-43c8-90a9-95dad3ae3edd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:11.955841 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c62a97-739e-43c8-90a9-95dad3ae3edd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:11.957259 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c62a97-739e-43c8-90a9-95dad3ae3edd None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:11.978137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 540/1080] 10.222.0.65 () {56 vars in 1133 bytes} [Tue May 10 19:59:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:11.979790 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4686a-9991-44de-a5e7-0ec3bbb35899 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:12.080420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4686a-9991-44de-a5e7-0ec3bbb35899 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:12.113251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78747c0a-8810-4b10-84c3-e8576354f33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:12.113893 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78747c0a-8810-4b10-84c3-e8576354f33e None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:12.114205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78747c0a-8810-4b10-84c3-e8576354f33e None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:12.114483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78747c0a-8810-4b10-84c3-e8576354f33e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:12.187538 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78747c0a-8810-4b10-84c3-e8576354f33e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:12.188995 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78747c0a-8810-4b10-84c3-e8576354f33e None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:12.193977 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95394130-0563-4d50-a184-0e545cac979f None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:12.194309 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95394130-0563-4d50-a184-0e545cac979f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:12.194619 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95394130-0563-4d50-a184-0e545cac979f None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:12.196087 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 541/1081] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 19:59:12.200863 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd7eae-90ef-4dcd-8ffe-e51853fa3590 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:12.201152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd7eae-90ef-4dcd-8ffe-e51853fa3590 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:12.201442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd7eae-90ef-4dcd-8ffe-e51853fa3590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:12.218023 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-86dd7eae-90ef-4dcd-8ffe-e51853fa3590 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:12.219243 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-86dd7eae-90ef-4dcd-8ffe-e51853fa3590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:12.285970 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 542/1082] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:59:12.290733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 541/1083] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:59:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 19:59:12.291984 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd7eae-90ef-4dcd-8ffe-e51853fa3590 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:12.308078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78747c0a-8810-4b10-84c3-e8576354f33e None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:12.389441 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd7eae-90ef-4dcd-8ffe-e51853fa3590 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:12.405569 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78747c0a-8810-4b10-84c3-e8576354f33e None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:12.423980 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3f09a-ece4-44ce-8aaf-9e13d7bde48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:12.424891 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3f09a-ece4-44ce-8aaf-9e13d7bde48e None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:12.425175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3f09a-ece4-44ce-8aaf-9e13d7bde48e None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:12.425449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3f09a-ece4-44ce-8aaf-9e13d7bde48e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:12.438532 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3f09a-ece4-44ce-8aaf-9e13d7bde48e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:12.439985 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3f09a-ece4-44ce-8aaf-9e13d7bde48e None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:12.440853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2fb24-9482-4004-8e62-eb3e0c11bfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:12.441507 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2fb24-9482-4004-8e62-eb3e0c11bfd0 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:12.441793 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2fb24-9482-4004-8e62-eb3e0c11bfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:12.442069 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2fb24-9482-4004-8e62-eb3e0c11bfd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:12.458645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-61b3f09a-ece4-44ce-8aaf-9e13d7bde48e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:59:12.458645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:12.458645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:12.458645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:12.458645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:12.458645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:12.458645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:12.458645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:12.458645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:12.458645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:12.458645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:12.458645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:12.458645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:59:12.458645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:59:12.458645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:59:12.459579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:59:12.459579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:12.459579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:12.459579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:12.459579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:12.459579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:12.459579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:12.459579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:12.459579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:59:12.459579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:12.459579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:12.459579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:12.459579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:12.459579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:12.459579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:12.459579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:12.459579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:12.460501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:59:12.460501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:59:12.460501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:59:12.460501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:59:12.462396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 543/1084] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:59:12.465452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3f09a-ece4-44ce-8aaf-9e13d7bde48e None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:12.510095 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2fb24-9482-4004-8e62-eb3e0c11bfd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:12.511574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2fb24-9482-4004-8e62-eb3e0c11bfd0 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:12.565614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3f09a-ece4-44ce-8aaf-9e13d7bde48e None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:12.604078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f767561-610c-4b46-b99c-d5cda378eaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:12.604732 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f767561-610c-4b46-b99c-d5cda378eaac None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:12.605014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f767561-610c-4b46-b99c-d5cda378eaac None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:12.605286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f767561-610c-4b46-b99c-d5cda378eaac None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:12.605999 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f767561-610c-4b46-b99c-d5cda378eaac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:12.606331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f767561-610c-4b46-b99c-d5cda378eaac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:12.607761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f767561-610c-4b46-b99c-d5cda378eaac None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:12.619075 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 544/1085] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:12.624401 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f767561-610c-4b46-b99c-d5cda378eaac None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:12.630260 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 542/1086] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:59:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 10 19:59:12.631834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2fb24-9482-4004-8e62-eb3e0c11bfd0 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:12.730981 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f767561-610c-4b46-b99c-d5cda378eaac None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:12.733419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2fb24-9482-4004-8e62-eb3e0c11bfd0 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:12.765516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3b0d0-5431-4722-86b8-6658ec230b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:12.766338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3b0d0-5431-4722-86b8-6658ec230b8d None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:12.766627 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3b0d0-5431-4722-86b8-6658ec230b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:12.766903 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3b0d0-5431-4722-86b8-6658ec230b8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:12.767609 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c23516-6ae7-4972-b967-45ad6b05f026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:12.768715 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c23516-6ae7-4972-b967-45ad6b05f026 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:12.769190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c23516-6ae7-4972-b967-45ad6b05f026 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:12.769919 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c23516-6ae7-4972-b967-45ad6b05f026 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:12.773797 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3b0d0-5431-4722-86b8-6658ec230b8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:12.775275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3b0d0-5431-4722-86b8-6658ec230b8d None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:12.781658 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c23516-6ae7-4972-b967-45ad6b05f026 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:12.783134 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c23516-6ae7-4972-b967-45ad6b05f026 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:12.790013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-05c3b0d0-5431-4722-86b8-6658ec230b8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:59:12.790013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:12.790013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:12.790013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:12.790013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:12.790013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:12.790013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:12.790013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:12.790013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:12.790013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:12.790013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:12.790013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:12.790013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 19:59:12.790013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 19:59:12.790013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 19:59:12.790960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 19:59:12.790960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:12.790960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:12.790960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:12.790960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:12.790960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:12.790960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:12.790960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:12.790960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:59:12.790960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:12.790960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:12.790960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:12.790960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:12.790960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:12.790960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:12.790960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:12.790960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:12.792031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 19:59:12.792031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 19:59:12.792031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:59:12.792031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:59:12.792031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:59:12.792031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:59:12.792031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 19:59:12.792031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:59:12.792031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 545/1087] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 19:59:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:59:12.796829 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3b0d0-5431-4722-86b8-6658ec230b8d None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:12.801664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-89c23516-6ae7-4972-b967-45ad6b05f026 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:59:12.801664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:12.801664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:12.801664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:12.801664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:12.801664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:12.801664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:12.801664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:12.801664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:12.801664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:12.801664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:12.801664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:12.801664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:59:12.801664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:59:12.801664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:59:12.802787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:59:12.802787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:12.802787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:12.802787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:12.802787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:12.802787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:12.802787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:12.802787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:12.802787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:59:12.802787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:12.802787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:12.802787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:12.802787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:12.802787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:12.802787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:12.802787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:12.802787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:12.804176 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 19:59:12.804176 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:59:12.804176 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 19:59:12.804176 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:12.804176 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 543/1088] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:59:12.808515 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c23516-6ae7-4972-b967-45ad6b05f026 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:12.902936 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3b0d0-5431-4722-86b8-6658ec230b8d None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:12.911539 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c23516-6ae7-4972-b967-45ad6b05f026 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:12.942736 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8757a-7897-4b8f-bf94-36b9bf9d7075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:12.943395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8757a-7897-4b8f-bf94-36b9bf9d7075 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:12.943522 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f30972-236e-49e5-900e-99d7e273b174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:12.943679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8757a-7897-4b8f-bf94-36b9bf9d7075 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:12.943951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8757a-7897-4b8f-bf94-36b9bf9d7075 None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:12.944050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f30972-236e-49e5-900e-99d7e273b174 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:12.944307 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f30972-236e-49e5-900e-99d7e273b174 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:12.944587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f30972-236e-49e5-900e-99d7e273b174 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:12.944744 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8757a-7897-4b8f-bf94-36b9bf9d7075 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:12.945054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8757a-7897-4b8f-bf94-36b9bf9d7075 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:12.945396 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f30972-236e-49e5-900e-99d7e273b174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:12.945698 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f30972-236e-49e5-900e-99d7e273b174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:12.946520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8757a-7897-4b8f-bf94-36b9bf9d7075 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:12.947063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f30972-236e-49e5-900e-99d7e273b174 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:12.970943 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 544/1089] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 19:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:12.971156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 546/1090] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 19:59:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:12.971254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af4836-dd51-4cc7-b5ec-f3716f81f383 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:12.971254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af4836-dd51-4cc7-b5ec-f3716f81f383 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:12.971254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af4836-dd51-4cc7-b5ec-f3716f81f383 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:12.971254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 545/1091] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:12.971579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8757a-7897-4b8f-bf94-36b9bf9d7075 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:12.971669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa963d-b7fa-44d9-9f76-60df477012d3 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:12.971669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa963d-b7fa-44d9-9f76-60df477012d3 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:12.971669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa963d-b7fa-44d9-9f76-60df477012d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:12.988959 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-15fa963d-b7fa-44d9-9f76-60df477012d3 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:12.990177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-15fa963d-b7fa-44d9-9f76-60df477012d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:13.060875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 546/1092] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:59:13.078451 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8757a-7897-4b8f-bf94-36b9bf9d7075 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:13.083588 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa963d-b7fa-44d9-9f76-60df477012d3 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:13.116658 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996899c5-b3f1-4ab4-9829-9282c537e0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:13.117329 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996899c5-b3f1-4ab4-9829-9282c537e0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:13.117618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996899c5-b3f1-4ab4-9829-9282c537e0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:13.117931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996899c5-b3f1-4ab4-9829-9282c537e0d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:13.118787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996899c5-b3f1-4ab4-9829-9282c537e0d6 None admin] RBAC: Adding query filter params (role.id=2cb62fd2d3a747ee8827a5d19d6f9f2e, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=ae922b2da4814d10bed1fa345700ac04) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:13.119107 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996899c5-b3f1-4ab4-9829-9282c537e0d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:13.120574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996899c5-b3f1-4ab4-9829-9282c537e0d6 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:13.130444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 547/1093] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:59:12 2022] GET /identity/v3/role_assignments?user.id=ae922b2da4814d10bed1fa345700ac04&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=2cb62fd2d3a747ee8827a5d19d6f9f2e => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:13.195624 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa963d-b7fa-44d9-9f76-60df477012d3 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:13.234975 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78edd002-b299-4cc8-a83c-fd9b7ca86663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:13.235790 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78edd002-b299-4cc8-a83c-fd9b7ca86663 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:13.236071 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78edd002-b299-4cc8-a83c-fd9b7ca86663 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:13.236613 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78edd002-b299-4cc8-a83c-fd9b7ca86663 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:13.248520 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78edd002-b299-4cc8-a83c-fd9b7ca86663 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:59:13.256159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-78edd002-b299-4cc8-a83c-fd9b7ca86663 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:59:13.256159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:13.256159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:13.256159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:13.256159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:13.256159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:13.256159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:13.256159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:13.256159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:13.256159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:13.256159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:13.256159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:13.256159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:59:13.256159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:59:13.256159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:59:13.256159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:59:13.257379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:59:13.257379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:59:13.257379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:59:13.257379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:59:13.257379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:13.257379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:13.257379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:13.257379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:13.257379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:13.257379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:13.257379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:13.257379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:59:13.257379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:13.257379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:13.257379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:13.257379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:13.258505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:13.258505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:13.258505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:13.258505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:13.258505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:59:13.258505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:59:13.258505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:59:13.258505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:59:13.258505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:59:13.258505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:59:13.258505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:59:13.258505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:13.258505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 547/1094] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:59:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 19:59:13.266216 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996899c5-b3f1-4ab4-9829-9282c537e0d6 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:13.370842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996899c5-b3f1-4ab4-9829-9282c537e0d6 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:13.405498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66eb6e-d5bb-4c4b-b6e8-7d021d2c3e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:13.406239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66eb6e-d5bb-4c4b-b6e8-7d021d2c3e3f None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:13.406518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66eb6e-d5bb-4c4b-b6e8-7d021d2c3e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:13.406791 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66eb6e-d5bb-4c4b-b6e8-7d021d2c3e3f None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:13.407545 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66eb6e-d5bb-4c4b-b6e8-7d021d2c3e3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:13.407850 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66eb6e-d5bb-4c4b-b6e8-7d021d2c3e3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:13.409212 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66eb6e-d5bb-4c4b-b6e8-7d021d2c3e3f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:13.417563 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 548/1095] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:59:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:13.423633 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78edd002-b299-4cc8-a83c-fd9b7ca86663 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:13.524540 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78edd002-b299-4cc8-a83c-fd9b7ca86663 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:13.558034 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f817ffd-f843-492f-a261-901ddf08bf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:13.558969 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f817ffd-f843-492f-a261-901ddf08bf78 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:13.559251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f817ffd-f843-492f-a261-901ddf08bf78 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:13.559520 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f817ffd-f843-492f-a261-901ddf08bf78 None admin] PATH_INFO: `/v3/users/b6dc9c6540fe48b6939c76e4368ecc4a` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:13.560702 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f817ffd-f843-492f-a261-901ddf08bf78 None admin] RBAC: Authorizing `identity:get_user(user_id=b6dc9c6540fe48b6939c76e4368ecc4a)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:13.562185 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f817ffd-f843-492f-a261-901ddf08bf78 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:13.564561 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 548/1096] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:59:13 2022] GET /identity/v3/users/b6dc9c6540fe48b6939c76e4368ecc4a => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:13.572930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66eb6e-d5bb-4c4b-b6e8-7d021d2c3e3f None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:13.673081 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66eb6e-d5bb-4c4b-b6e8-7d021d2c3e3f None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:13.707657 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade47668-dbcd-49b9-ada3-7a857f0841aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:13.708516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade47668-dbcd-49b9-ada3-7a857f0841aa None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:13.708797 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade47668-dbcd-49b9-ada3-7a857f0841aa None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:13.709071 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade47668-dbcd-49b9-ada3-7a857f0841aa None admin] PATH_INFO: `/v3/projects/58e0c54a4af644aba7536199faf19434` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:13.710292 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade47668-dbcd-49b9-ada3-7a857f0841aa None admin] RBAC: Authorizing `identity:get_project(project_id=58e0c54a4af644aba7536199faf19434)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:13.711715 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade47668-dbcd-49b9-ada3-7a857f0841aa None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:13.713413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 549/1097] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:59:13 2022] GET /identity/v3/projects/58e0c54a4af644aba7536199faf19434 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:13.719437 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f817ffd-f843-492f-a261-901ddf08bf78 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:13.818820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f817ffd-f843-492f-a261-901ddf08bf78 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:13.851385 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad993c-898a-4dad-8d75-c5351c806ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:13.852266 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad993c-898a-4dad-8d75-c5351c806ade None admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:13.852543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad993c-898a-4dad-8d75-c5351c806ade None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:13.852815 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad993c-898a-4dad-8d75-c5351c806ade None admin] PATH_INFO: `/v3/projects/58e0c54a4af644aba7536199faf19434/users/b6dc9c6540fe48b6939c76e4368ecc4a/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:13.854783 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad993c-898a-4dad-8d75-c5351c806ade None admin] RBAC: Authorizing `identity:create_grant(project_id=58e0c54a4af644aba7536199faf19434, user_id=b6dc9c6540fe48b6939c76e4368ecc4a, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:13.856208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad993c-898a-4dad-8d75-c5351c806ade None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:13.934388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 549/1098] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:59:13 2022] PUT /identity/v3/projects/58e0c54a4af644aba7536199faf19434/users/b6dc9c6540fe48b6939c76e4368ecc4a/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 19:59:16.114878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c366e5-d6ab-41cb-b597-4f9cac3d0525 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:16.115462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c366e5-d6ab-41cb-b597-4f9cac3d0525 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:16.115462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c366e5-d6ab-41cb-b597-4f9cac3d0525 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:16.116812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 550/1099] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:16.122017 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879bd9e-f219-43c2-abca-bc399fc71a52 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:16.122421 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879bd9e-f219-43c2-abca-bc399fc71a52 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:16.122597 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879bd9e-f219-43c2-abca-bc399fc71a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:16.138762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-3879bd9e-f219-43c2-abca-bc399fc71a52 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:16.139945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3879bd9e-f219-43c2-abca-bc399fc71a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:16.202004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 550/1100] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:59:16.225201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c366e5-d6ab-41cb-b597-4f9cac3d0525 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:16.324518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c366e5-d6ab-41cb-b597-4f9cac3d0525 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:16.356332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae7ca9-2e12-4ea2-9d23-72a117948d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:16.357111 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae7ca9-2e12-4ea2-9d23-72a117948d3e None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:16.357392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae7ca9-2e12-4ea2-9d23-72a117948d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:16.357700 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae7ca9-2e12-4ea2-9d23-72a117948d3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:16.368009 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae7ca9-2e12-4ea2-9d23-72a117948d3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:59:16.375468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-36ae7ca9-2e12-4ea2-9d23-72a117948d3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:59:16.375468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:16.375468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:16.375468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:16.375468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:16.375468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:16.375468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:16.375468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:16.375468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:16.375468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:16.375468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:16.375468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:16.375468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:59:16.375468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:59:16.375468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:59:16.375468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:59:16.376634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:59:16.376634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:59:16.376634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:59:16.376634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:59:16.376634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:16.376634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:16.376634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:16.376634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:16.376634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:16.376634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:16.376634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:16.376634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:59:16.376634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:16.376634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:16.376634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:16.376634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:16.377864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:16.377864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:16.377864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:16.377864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:16.377864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:59:16.377864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:59:16.377864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:59:16.377864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:59:16.377864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:59:16.377864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:59:16.377864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:59:16.377864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:59:16.377864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 551/1101] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:59:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:59:16.382661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879bd9e-f219-43c2-abca-bc399fc71a52 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:16.476214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879bd9e-f219-43c2-abca-bc399fc71a52 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:16.508648 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2f7c1-2148-4b79-84f5-4891b7b2782b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:16.509938 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2f7c1-2148-4b79-84f5-4891b7b2782b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:16.510252 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2f7c1-2148-4b79-84f5-4891b7b2782b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:16.510527 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2f7c1-2148-4b79-84f5-4891b7b2782b None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:16.511343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b2f7c1-2148-4b79-84f5-4891b7b2782b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:16.511678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b2f7c1-2148-4b79-84f5-4891b7b2782b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:16.513038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b2f7c1-2148-4b79-84f5-4891b7b2782b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:16.521262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 551/1102] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:59:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:16.527023 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae7ca9-2e12-4ea2-9d23-72a117948d3e None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:16.677860 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae7ca9-2e12-4ea2-9d23-72a117948d3e None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:16.709944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb63842-5357-4aac-bea1-301e5c95c32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:16.710817 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb63842-5357-4aac-bea1-301e5c95c32c None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:16.711118 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb63842-5357-4aac-bea1-301e5c95c32c None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:16.711391 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb63842-5357-4aac-bea1-301e5c95c32c None admin] PATH_INFO: `/v3/users/b6dc9c6540fe48b6939c76e4368ecc4a` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:16.712551 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb63842-5357-4aac-bea1-301e5c95c32c None admin] RBAC: Authorizing `identity:get_user(user_id=b6dc9c6540fe48b6939c76e4368ecc4a)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:16.713969 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb63842-5357-4aac-bea1-301e5c95c32c None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:16.715596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 552/1103] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:59:16 2022] GET /identity/v3/users/b6dc9c6540fe48b6939c76e4368ecc4a => generated 285 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:16.721265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2f7c1-2148-4b79-84f5-4891b7b2782b None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:16.814254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2f7c1-2148-4b79-84f5-4891b7b2782b None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:16.845206 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754872e5-046e-4ce4-bae4-82ea501da196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:16.845850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754872e5-046e-4ce4-bae4-82ea501da196 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:16.846152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754872e5-046e-4ce4-bae4-82ea501da196 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:16.846424 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754872e5-046e-4ce4-bae4-82ea501da196 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:16.921714 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754872e5-046e-4ce4-bae4-82ea501da196 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:16.923186 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754872e5-046e-4ce4-bae4-82ea501da196 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:17.019368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 552/1104] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:59:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 19:59:17.025251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb63842-5357-4aac-bea1-301e5c95c32c None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:17.118352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb63842-5357-4aac-bea1-301e5c95c32c None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:17.150842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec199e-e75b-43ee-ade7-6b9292fc252d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:17.151733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec199e-e75b-43ee-ade7-6b9292fc252d None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:17.152019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec199e-e75b-43ee-ade7-6b9292fc252d None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:17.152296 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec199e-e75b-43ee-ade7-6b9292fc252d None admin] PATH_INFO: `/v3/projects/58e0c54a4af644aba7536199faf19434` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:17.153471 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec199e-e75b-43ee-ade7-6b9292fc252d None admin] RBAC: Authorizing `identity:get_project(project_id=58e0c54a4af644aba7536199faf19434)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:17.154924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec199e-e75b-43ee-ade7-6b9292fc252d None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:17.156588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 553/1105] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:59:17 2022] GET /identity/v3/projects/58e0c54a4af644aba7536199faf19434 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:17.162709 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754872e5-046e-4ce4-bae4-82ea501da196 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:17.256337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754872e5-046e-4ce4-bae4-82ea501da196 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:17.287945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477bc72-9339-4e23-a115-aef5de4b9ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:17.288590 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477bc72-9339-4e23-a115-aef5de4b9ecf None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:17.288867 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477bc72-9339-4e23-a115-aef5de4b9ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:17.289135 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477bc72-9339-4e23-a115-aef5de4b9ecf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:17.289960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477bc72-9339-4e23-a115-aef5de4b9ecf None admin] RBAC: Adding query filter params (role.id=5c87e804d0764309ba08ad6e06b23a87, scope.project.id=58e0c54a4af644aba7536199faf19434, user.id=b6dc9c6540fe48b6939c76e4368ecc4a) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:17.290294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477bc72-9339-4e23-a115-aef5de4b9ecf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:17.291803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477bc72-9339-4e23-a115-aef5de4b9ecf None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:17.297846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 553/1106] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:59:17 2022] GET /identity/v3/role_assignments?user.id=b6dc9c6540fe48b6939c76e4368ecc4a&scope.project.id=58e0c54a4af644aba7536199faf19434&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:19.495530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd79722-8bbe-4ff5-8ca9-47a149d860c2 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:19.496024 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd79722-8bbe-4ff5-8ca9-47a149d860c2 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:19.496125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd79722-8bbe-4ff5-8ca9-47a149d860c2 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:19.497453 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 554/1107] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:19.503406 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea2022-9982-415d-94b6-4711b1cb2aaf None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:19.503818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea2022-9982-415d-94b6-4711b1cb2aaf None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:19.503957 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea2022-9982-415d-94b6-4711b1cb2aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:19.520729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-34ea2022-9982-415d-94b6-4711b1cb2aaf None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:19.521880 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-34ea2022-9982-415d-94b6-4711b1cb2aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:19.584110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 554/1108] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:59:19.609857 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd79722-8bbe-4ff5-8ca9-47a149d860c2 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:19.714853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd79722-8bbe-4ff5-8ca9-47a149d860c2 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:19.748753 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dee5e-501f-4772-a62a-8ecbe746cae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:19.749520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dee5e-501f-4772-a62a-8ecbe746cae1 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:19.749803 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dee5e-501f-4772-a62a-8ecbe746cae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:19.750102 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dee5e-501f-4772-a62a-8ecbe746cae1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:19.751274 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59dee5e-501f-4772-a62a-8ecbe746cae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:19.752693 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59dee5e-501f-4772-a62a-8ecbe746cae1 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:19.754296 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 555/1109] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:59:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:19.759966 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea2022-9982-415d-94b6-4711b1cb2aaf None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:19.851453 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea2022-9982-415d-94b6-4711b1cb2aaf None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:19.883202 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d44fb5-df47-463c-8b54-eb85a6c082fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:19.883843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d44fb5-df47-463c-8b54-eb85a6c082fc None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:19.884121 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d44fb5-df47-463c-8b54-eb85a6c082fc None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:19.884388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d44fb5-df47-463c-8b54-eb85a6c082fc None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:19.885092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d44fb5-df47-463c-8b54-eb85a6c082fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:19.886529 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d44fb5-df47-463c-8b54-eb85a6c082fc None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:19.990865 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 555/1110] 10.222.0.65 () {60 vars in 1132 bytes} [Tue May 10 19:59:19 2022] POST /identity/v3/users => generated 286 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:59:22.208111 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0223a-3675-43e6-b3ee-d5abed14b9dc None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:22.208666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0223a-3675-43e6-b3ee-d5abed14b9dc None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:22.208666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0223a-3675-43e6-b3ee-d5abed14b9dc None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:22.210025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 556/1111] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:22.215333 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986a574-65c2-4467-a22a-cb63fbe28b9d None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:22.215762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986a574-65c2-4467-a22a-cb63fbe28b9d None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:22.215901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986a574-65c2-4467-a22a-cb63fbe28b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:22.232632 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-4986a574-65c2-4467-a22a-cb63fbe28b9d None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:22.233804 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4986a574-65c2-4467-a22a-cb63fbe28b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:22.297378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 556/1112] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:59:22.320230 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0223a-3675-43e6-b3ee-d5abed14b9dc None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:22.416512 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0223a-3675-43e6-b3ee-d5abed14b9dc None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:22.448816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baad40a-57c0-41ee-8859-7f9840f54917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:22.449656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baad40a-57c0-41ee-8859-7f9840f54917 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:22.449940 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baad40a-57c0-41ee-8859-7f9840f54917 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:22.450252 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baad40a-57c0-41ee-8859-7f9840f54917 None admin] PATH_INFO: `/v3/roles/4fbb0a346f5f4f36bb32664ee799db42` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:22.452253 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baad40a-57c0-41ee-8859-7f9840f54917 None admin] RBAC: Authorizing `identity:get_role(role_id=4fbb0a346f5f4f36bb32664ee799db42)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:22.453612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baad40a-57c0-41ee-8859-7f9840f54917 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:22.455005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 557/1113] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:59:22 2022] GET /identity/v3/roles/4fbb0a346f5f4f36bb32664ee799db42 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:22.460739 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986a574-65c2-4467-a22a-cb63fbe28b9d None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:22.553423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986a574-65c2-4467-a22a-cb63fbe28b9d None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:22.585170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12606c-b79a-4c06-b3b7-d30a08e6220c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:22.586015 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12606c-b79a-4c06-b3b7-d30a08e6220c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:22.586328 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12606c-b79a-4c06-b3b7-d30a08e6220c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:22.586605 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12606c-b79a-4c06-b3b7-d30a08e6220c None admin] PATH_INFO: `/v3/users/696c3971c684427baebf65f8a788b835` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:22.599464 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12606c-b79a-4c06-b3b7-d30a08e6220c None admin] RBAC: Authorizing `identity:get_user(user_id=696c3971c684427baebf65f8a788b835)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:22.600899 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12606c-b79a-4c06-b3b7-d30a08e6220c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:22.602553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 557/1114] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:59:22 2022] GET /identity/v3/users/696c3971c684427baebf65f8a788b835 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:22.608048 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baad40a-57c0-41ee-8859-7f9840f54917 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:22.701392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baad40a-57c0-41ee-8859-7f9840f54917 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:22.733854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10741efd-c715-48f4-b67f-cd207047789d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:22.734456 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10741efd-c715-48f4-b67f-cd207047789d None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:22.734742 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10741efd-c715-48f4-b67f-cd207047789d None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:22.735017 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10741efd-c715-48f4-b67f-cd207047789d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:22.798253 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10741efd-c715-48f4-b67f-cd207047789d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:22.799681 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10741efd-c715-48f4-b67f-cd207047789d None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:22.890791 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 558/1115] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:59:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 19:59:22.896771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12606c-b79a-4c06-b3b7-d30a08e6220c None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:22.991040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12606c-b79a-4c06-b3b7-d30a08e6220c None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:23.022832 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f92353-4661-4e30-815a-44feb40672e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:23.023666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f92353-4661-4e30-815a-44feb40672e5 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:23.023949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f92353-4661-4e30-815a-44feb40672e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:23.024222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f92353-4661-4e30-815a-44feb40672e5 None admin] PATH_INFO: `/v3/projects/58e0c54a4af644aba7536199faf19434` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:23.025400 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f92353-4661-4e30-815a-44feb40672e5 None admin] RBAC: Authorizing `identity:get_project(project_id=58e0c54a4af644aba7536199faf19434)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:23.026839 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f92353-4661-4e30-815a-44feb40672e5 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:23.028486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 558/1116] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:59:22 2022] GET /identity/v3/projects/58e0c54a4af644aba7536199faf19434 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:23.034931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10741efd-c715-48f4-b67f-cd207047789d None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:23.127189 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10741efd-c715-48f4-b67f-cd207047789d None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:23.159551 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e7c60-dd6f-47e2-983a-d53f8d2047b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:23.160187 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e7c60-dd6f-47e2-983a-d53f8d2047b5 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:23.160471 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e7c60-dd6f-47e2-983a-d53f8d2047b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:23.160748 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e7c60-dd6f-47e2-983a-d53f8d2047b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:23.161582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e7c60-dd6f-47e2-983a-d53f8d2047b5 None admin] RBAC: Adding query filter params (role.id=4fbb0a346f5f4f36bb32664ee799db42, scope.project.id=58e0c54a4af644aba7536199faf19434, user.id=696c3971c684427baebf65f8a788b835) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:23.161894 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e7c60-dd6f-47e2-983a-d53f8d2047b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:23.163335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e7c60-dd6f-47e2-983a-d53f8d2047b5 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:23.168557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 559/1117] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:59:23 2022] GET /identity/v3/role_assignments?user.id=696c3971c684427baebf65f8a788b835&scope.project.id=58e0c54a4af644aba7536199faf19434&role.id=4fbb0a346f5f4f36bb32664ee799db42 => generated 259 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:25.330370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0e556-ed7b-44b1-8f9f-faa1f463da2b None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:25.330900 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0e556-ed7b-44b1-8f9f-faa1f463da2b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:25.330900 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0e556-ed7b-44b1-8f9f-faa1f463da2b None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:25.332256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 559/1118] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:25.337274 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16384c2e-9150-4b23-b6e2-5fd3edba3450 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:25.337681 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16384c2e-9150-4b23-b6e2-5fd3edba3450 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:25.337819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16384c2e-9150-4b23-b6e2-5fd3edba3450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:25.353896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-16384c2e-9150-4b23-b6e2-5fd3edba3450 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:25.355079 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-16384c2e-9150-4b23-b6e2-5fd3edba3450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:25.416945 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 560/1119] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:59:25.439991 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0e556-ed7b-44b1-8f9f-faa1f463da2b None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:25.535084 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0e556-ed7b-44b1-8f9f-faa1f463da2b None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:25.566718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc49bb-6d06-40ed-aa7d-81214ddba43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:25.567554 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc49bb-6d06-40ed-aa7d-81214ddba43f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:25.567841 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc49bb-6d06-40ed-aa7d-81214ddba43f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:25.568198 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc49bb-6d06-40ed-aa7d-81214ddba43f None admin] PATH_INFO: `/v3/roles/4fbb0a346f5f4f36bb32664ee799db42` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:25.569642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc49bb-6d06-40ed-aa7d-81214ddba43f None admin] RBAC: Authorizing `identity:get_role(role_id=4fbb0a346f5f4f36bb32664ee799db42)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:25.571035 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc49bb-6d06-40ed-aa7d-81214ddba43f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:25.572386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 560/1120] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:59:25 2022] GET /identity/v3/roles/4fbb0a346f5f4f36bb32664ee799db42 => generated 223 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:25.578795 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16384c2e-9150-4b23-b6e2-5fd3edba3450 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:25.673899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16384c2e-9150-4b23-b6e2-5fd3edba3450 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:25.705145 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f8a4b-e18e-47d5-8e86-00a71a90c015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:25.705998 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f8a4b-e18e-47d5-8e86-00a71a90c015 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:25.706303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f8a4b-e18e-47d5-8e86-00a71a90c015 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:25.706575 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f8a4b-e18e-47d5-8e86-00a71a90c015 None admin] PATH_INFO: `/v3/users/696c3971c684427baebf65f8a788b835` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:25.707738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f8a4b-e18e-47d5-8e86-00a71a90c015 None admin] RBAC: Authorizing `identity:get_user(user_id=696c3971c684427baebf65f8a788b835)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:25.709160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f8a4b-e18e-47d5-8e86-00a71a90c015 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:25.710871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 561/1121] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:59:25 2022] GET /identity/v3/users/696c3971c684427baebf65f8a788b835 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:25.717100 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc49bb-6d06-40ed-aa7d-81214ddba43f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:25.811668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc49bb-6d06-40ed-aa7d-81214ddba43f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:25.843265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0f657-e32a-442f-a76b-267f6bd66960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:25.844095 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0f657-e32a-442f-a76b-267f6bd66960 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:25.844378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0f657-e32a-442f-a76b-267f6bd66960 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:25.844654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0f657-e32a-442f-a76b-267f6bd66960 None admin] PATH_INFO: `/v3/projects/58e0c54a4af644aba7536199faf19434` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:25.845819 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0f657-e32a-442f-a76b-267f6bd66960 None admin] RBAC: Authorizing `identity:get_project(project_id=58e0c54a4af644aba7536199faf19434)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:25.847267 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0f657-e32a-442f-a76b-267f6bd66960 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:25.848914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 561/1122] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:59:25 2022] GET /identity/v3/projects/58e0c54a4af644aba7536199faf19434 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:25.854565 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f8a4b-e18e-47d5-8e86-00a71a90c015 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:25.947246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f8a4b-e18e-47d5-8e86-00a71a90c015 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:25.979661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1540ac0-d4f1-4a20-b9a9-db662d646acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:25.980514 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1540ac0-d4f1-4a20-b9a9-db662d646acb None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:25.980791 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1540ac0-d4f1-4a20-b9a9-db662d646acb None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:25.981059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1540ac0-d4f1-4a20-b9a9-db662d646acb None admin] PATH_INFO: `/v3/projects/58e0c54a4af644aba7536199faf19434/users/696c3971c684427baebf65f8a788b835/roles/4fbb0a346f5f4f36bb32664ee799db42` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:25.983255 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1540ac0-d4f1-4a20-b9a9-db662d646acb None admin] RBAC: Authorizing `identity:create_grant(project_id=58e0c54a4af644aba7536199faf19434, user_id=696c3971c684427baebf65f8a788b835, role_id=4fbb0a346f5f4f36bb32664ee799db42)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:25.984654 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1540ac0-d4f1-4a20-b9a9-db662d646acb None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:26.070840 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 562/1123] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:59:25 2022] PUT /identity/v3/projects/58e0c54a4af644aba7536199faf19434/users/696c3971c684427baebf65f8a788b835/roles/4fbb0a346f5f4f36bb32664ee799db42 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 19:59:28.239975 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023f78a-89a3-463c-b138-d751db99a927 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:28.240505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023f78a-89a3-463c-b138-d751db99a927 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:28.240598 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023f78a-89a3-463c-b138-d751db99a927 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:28.241891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 562/1124] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:28.247161 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4bbf6-53ce-4aa1-83e7-8ff6fbb2fca6 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:28.248161 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4bbf6-53ce-4aa1-83e7-8ff6fbb2fca6 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:28.248305 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4bbf6-53ce-4aa1-83e7-8ff6fbb2fca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:28.264428 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-c9b4bbf6-53ce-4aa1-83e7-8ff6fbb2fca6 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:28.265609 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c9b4bbf6-53ce-4aa1-83e7-8ff6fbb2fca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:28.327683 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 563/1125] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:59:28.351111 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023f78a-89a3-463c-b138-d751db99a927 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:28.450812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023f78a-89a3-463c-b138-d751db99a927 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:28.481621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e0563-6318-4333-b42b-860de4fb8ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:28.482989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e0563-6318-4333-b42b-860de4fb8ebe None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:28.483266 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e0563-6318-4333-b42b-860de4fb8ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:28.483539 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e0563-6318-4333-b42b-860de4fb8ebe None admin] PATH_INFO: `/v3/roles/4fbb0a346f5f4f36bb32664ee799db42` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:28.484981 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e0563-6318-4333-b42b-860de4fb8ebe None admin] RBAC: Authorizing `identity:get_role(role_id=4fbb0a346f5f4f36bb32664ee799db42)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:28.486633 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e0563-6318-4333-b42b-860de4fb8ebe None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:28.487969 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 563/1126] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:59:28 2022] GET /identity/v3/roles/4fbb0a346f5f4f36bb32664ee799db42 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:28.493743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4bbf6-53ce-4aa1-83e7-8ff6fbb2fca6 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:28.585650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4bbf6-53ce-4aa1-83e7-8ff6fbb2fca6 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:28.619611 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f708a-20d9-47dd-bc23-61243e24c0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:28.620462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f708a-20d9-47dd-bc23-61243e24c0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:28.620746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f708a-20d9-47dd-bc23-61243e24c0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:28.621018 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f708a-20d9-47dd-bc23-61243e24c0f5 None admin] PATH_INFO: `/v3/users/696c3971c684427baebf65f8a788b835` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:28.622226 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f708a-20d9-47dd-bc23-61243e24c0f5 None admin] RBAC: Authorizing `identity:get_user(user_id=696c3971c684427baebf65f8a788b835)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:28.623779 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f708a-20d9-47dd-bc23-61243e24c0f5 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:28.625370 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 564/1127] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:59:28 2022] GET /identity/v3/users/696c3971c684427baebf65f8a788b835 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:28.630989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e0563-6318-4333-b42b-860de4fb8ebe None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:28.729137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e0563-6318-4333-b42b-860de4fb8ebe None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:28.761349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150e4b1-58da-45a4-bfbd-94952a23470f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:28.761927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150e4b1-58da-45a4-bfbd-94952a23470f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:28.762244 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150e4b1-58da-45a4-bfbd-94952a23470f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:28.762524 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150e4b1-58da-45a4-bfbd-94952a23470f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:28.825130 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150e4b1-58da-45a4-bfbd-94952a23470f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:28.826635 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150e4b1-58da-45a4-bfbd-94952a23470f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:28.917978 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 564/1128] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:59:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 19:59:28.924398 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f708a-20d9-47dd-bc23-61243e24c0f5 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:29.016994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f708a-20d9-47dd-bc23-61243e24c0f5 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:29.048633 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8bc7e-c566-440e-848b-009c13d34139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:29.049467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8bc7e-c566-440e-848b-009c13d34139 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:29.049748 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8bc7e-c566-440e-848b-009c13d34139 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:29.050020 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8bc7e-c566-440e-848b-009c13d34139 None admin] PATH_INFO: `/v3/projects/58e0c54a4af644aba7536199faf19434` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:29.051208 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d8bc7e-c566-440e-848b-009c13d34139 None admin] RBAC: Authorizing `identity:get_project(project_id=58e0c54a4af644aba7536199faf19434)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:29.052621 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d8bc7e-c566-440e-848b-009c13d34139 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:29.054364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 565/1129] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:59:28 2022] GET /identity/v3/projects/58e0c54a4af644aba7536199faf19434 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:29.060201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150e4b1-58da-45a4-bfbd-94952a23470f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:29.152277 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150e4b1-58da-45a4-bfbd-94952a23470f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:29.184641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673c462-1b5c-431f-8c1b-4562bdade447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:29.185334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673c462-1b5c-431f-8c1b-4562bdade447 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:29.185617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673c462-1b5c-431f-8c1b-4562bdade447 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:29.185891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673c462-1b5c-431f-8c1b-4562bdade447 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:29.186757 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673c462-1b5c-431f-8c1b-4562bdade447 None admin] RBAC: Adding query filter params (role.id=4fbb0a346f5f4f36bb32664ee799db42, scope.project.id=58e0c54a4af644aba7536199faf19434, user.id=696c3971c684427baebf65f8a788b835) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:29.187071 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673c462-1b5c-431f-8c1b-4562bdade447 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:29.188475 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673c462-1b5c-431f-8c1b-4562bdade447 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:29.194047 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 565/1130] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:59:29 2022] GET /identity/v3/role_assignments?user.id=696c3971c684427baebf65f8a788b835&scope.project.id=58e0c54a4af644aba7536199faf19434&role.id=4fbb0a346f5f4f36bb32664ee799db42 => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:31.420875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d069ebf-3d78-4800-a136-6f94fdb42e7c None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:31.421400 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d069ebf-3d78-4800-a136-6f94fdb42e7c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:31.421400 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d069ebf-3d78-4800-a136-6f94fdb42e7c None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:31.422800 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 566/1131] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:31.428091 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75867f-4ca0-4936-a1e1-97add034b099 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:31.428532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75867f-4ca0-4936-a1e1-97add034b099 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:31.428657 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75867f-4ca0-4936-a1e1-97add034b099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:31.445591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-9f75867f-4ca0-4936-a1e1-97add034b099 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:31.446827 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-9f75867f-4ca0-4936-a1e1-97add034b099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:31.509793 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 566/1132] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:59:31.533471 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d069ebf-3d78-4800-a136-6f94fdb42e7c None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:31.629392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d069ebf-3d78-4800-a136-6f94fdb42e7c None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:31.662821 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98aaec7-16ff-40a1-a05c-77f1cd59dbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:31.663600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98aaec7-16ff-40a1-a05c-77f1cd59dbd5 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:31.663883 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98aaec7-16ff-40a1-a05c-77f1cd59dbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:31.664157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98aaec7-16ff-40a1-a05c-77f1cd59dbd5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:31.665345 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98aaec7-16ff-40a1-a05c-77f1cd59dbd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:31.666790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98aaec7-16ff-40a1-a05c-77f1cd59dbd5 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:31.668740 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 567/1133] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:59:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:31.674600 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75867f-4ca0-4936-a1e1-97add034b099 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:31.768917 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75867f-4ca0-4936-a1e1-97add034b099 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:31.801476 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe97fe7-c5e1-4d0c-b9ab-e05b20b3f111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:31.802148 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe97fe7-c5e1-4d0c-b9ab-e05b20b3f111 None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:31.802435 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe97fe7-c5e1-4d0c-b9ab-e05b20b3f111 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:31.802713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe97fe7-c5e1-4d0c-b9ab-e05b20b3f111 None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:31.803423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe97fe7-c5e1-4d0c-b9ab-e05b20b3f111 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:31.804836 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe97fe7-c5e1-4d0c-b9ab-e05b20b3f111 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:31.895401 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 567/1134] 10.222.0.65 () {60 vars in 1141 bytes} [Tue May 10 19:59:31 2022] POST /identity/v3/projects => generated 311 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:59:34.109881 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bf1f9-7288-4672-b1ec-0c6bbf8a15ed None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:34.110682 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bf1f9-7288-4672-b1ec-0c6bbf8a15ed None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:34.110773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bf1f9-7288-4672-b1ec-0c6bbf8a15ed None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:34.112183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 568/1135] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:34.117522 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f5021-5af6-4681-8ba1-d1f3cacc3696 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:34.117939 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f5021-5af6-4681-8ba1-d1f3cacc3696 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:34.118106 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f5021-5af6-4681-8ba1-d1f3cacc3696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:34.134360 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-415f5021-5af6-4681-8ba1-d1f3cacc3696 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:34.135515 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-415f5021-5af6-4681-8ba1-d1f3cacc3696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:34.197615 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 568/1136] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:59:34.222819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8bf1f9-7288-4672-b1ec-0c6bbf8a15ed None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:34.320699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8bf1f9-7288-4672-b1ec-0c6bbf8a15ed None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:34.351886 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcba902-9bf8-4e84-8775-3d185ffa7254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:34.352659 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcba902-9bf8-4e84-8775-3d185ffa7254 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:34.352944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcba902-9bf8-4e84-8775-3d185ffa7254 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:34.353221 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcba902-9bf8-4e84-8775-3d185ffa7254 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:34.354442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcba902-9bf8-4e84-8775-3d185ffa7254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:34.355867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcba902-9bf8-4e84-8775-3d185ffa7254 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:34.357953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 569/1137] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 19:59:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:34.363845 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f5021-5af6-4681-8ba1-d1f3cacc3696 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:34.456942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f5021-5af6-4681-8ba1-d1f3cacc3696 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:34.488904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d9001-89d7-46cc-90d5-482ed8d86ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:34.489565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d9001-89d7-46cc-90d5-482ed8d86ca2 None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:34.489852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d9001-89d7-46cc-90d5-482ed8d86ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:34.490156 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d9001-89d7-46cc-90d5-482ed8d86ca2 None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:34.490866 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d9001-89d7-46cc-90d5-482ed8d86ca2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:34.492295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d9001-89d7-46cc-90d5-482ed8d86ca2 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:34.589798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 569/1138] 10.222.0.65 () {60 vars in 1132 bytes} [Tue May 10 19:59:34 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:59:36.822027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb037956-adcc-4a2c-a2de-9af11684cefe None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:36.822581 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb037956-adcc-4a2c-a2de-9af11684cefe None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:36.822581 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb037956-adcc-4a2c-a2de-9af11684cefe None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:36.823960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 570/1139] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:36.829231 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dab4b6-d276-4df7-8e39-51a84acef297 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:36.829668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dab4b6-d276-4df7-8e39-51a84acef297 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:36.829774 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dab4b6-d276-4df7-8e39-51a84acef297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:36.846367 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-84dab4b6-d276-4df7-8e39-51a84acef297 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:36.847535 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-84dab4b6-d276-4df7-8e39-51a84acef297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:36.909090 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 570/1140] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:59:36.932090 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb037956-adcc-4a2c-a2de-9af11684cefe None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:37.026294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb037956-adcc-4a2c-a2de-9af11684cefe None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:37.058556 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ba89f-5a61-462d-841c-7555700d0188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:37.059362 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ba89f-5a61-462d-841c-7555700d0188 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:37.059646 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ba89f-5a61-462d-841c-7555700d0188 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:37.059924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ba89f-5a61-462d-841c-7555700d0188 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:37.069527 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ba89f-5a61-462d-841c-7555700d0188 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:59:37.077196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-ce7ba89f-5a61-462d-841c-7555700d0188 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:59:37.077196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:37.077196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:37.077196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:37.077196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:37.077196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:37.077196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:37.077196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:37.077196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:37.077196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:37.077196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:37.077196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:37.077196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:59:37.077196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:59:37.077196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:59:37.077196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:59:37.078353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:59:37.078353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:59:37.078353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:59:37.078353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:59:37.078353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:37.078353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:37.078353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:37.078353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:37.078353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:37.078353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:37.078353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:37.078353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:59:37.078353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:37.078353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:37.078353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:37.078353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:37.079413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:37.079413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:37.079413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:37.079413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:37.079413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:59:37.079413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:59:37.079413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:59:37.079413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:59:37.079413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:59:37.079413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:59:37.079413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:59:37.079413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:59:37.079413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 571/1141] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:59:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:59:37.084208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dab4b6-d276-4df7-8e39-51a84acef297 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:37.177168 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dab4b6-d276-4df7-8e39-51a84acef297 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:37.209095 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531e26e-c7dd-4d2d-accf-1e75f69cf07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:37.209737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531e26e-c7dd-4d2d-accf-1e75f69cf07b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:37.210021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531e26e-c7dd-4d2d-accf-1e75f69cf07b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:37.210326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531e26e-c7dd-4d2d-accf-1e75f69cf07b None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:37.211073 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531e26e-c7dd-4d2d-accf-1e75f69cf07b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:37.211381 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531e26e-c7dd-4d2d-accf-1e75f69cf07b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:37.212755 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531e26e-c7dd-4d2d-accf-1e75f69cf07b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:37.221038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 571/1142] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:59:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:37.226756 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ba89f-5a61-462d-841c-7555700d0188 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:37.321240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ba89f-5a61-462d-841c-7555700d0188 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:37.354037 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473570ad-978a-460e-8e81-689aaf69dd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:37.354865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473570ad-978a-460e-8e81-689aaf69dd9f None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:37.355150 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473570ad-978a-460e-8e81-689aaf69dd9f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:37.355426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473570ad-978a-460e-8e81-689aaf69dd9f None admin] PATH_INFO: `/v3/users/2f8bd0fe63884340b54d21651ddf7313` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:37.368220 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473570ad-978a-460e-8e81-689aaf69dd9f None admin] RBAC: Authorizing `identity:get_user(user_id=2f8bd0fe63884340b54d21651ddf7313)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:37.369655 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473570ad-978a-460e-8e81-689aaf69dd9f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:37.371306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 572/1143] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:59:37 2022] GET /identity/v3/users/2f8bd0fe63884340b54d21651ddf7313 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:37.376838 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531e26e-c7dd-4d2d-accf-1e75f69cf07b None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:37.471824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531e26e-c7dd-4d2d-accf-1e75f69cf07b None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:37.502426 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964d048-7927-475c-a019-265d76f40096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:37.503058 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964d048-7927-475c-a019-265d76f40096 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:37.503338 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964d048-7927-475c-a019-265d76f40096 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:37.503610 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964d048-7927-475c-a019-265d76f40096 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:37.565360 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964d048-7927-475c-a019-265d76f40096 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:37.566801 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964d048-7927-475c-a019-265d76f40096 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:37.657864 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 572/1144] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:59:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 19:59:37.663687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473570ad-978a-460e-8e81-689aaf69dd9f None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:37.762222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473570ad-978a-460e-8e81-689aaf69dd9f None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:37.793876 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35715f7-06a0-49ef-a66f-cd539bfa5d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:37.794744 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35715f7-06a0-49ef-a66f-cd539bfa5d16 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:37.795030 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35715f7-06a0-49ef-a66f-cd539bfa5d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:37.795331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35715f7-06a0-49ef-a66f-cd539bfa5d16 None admin] PATH_INFO: `/v3/projects/a384e760046041c391b684c018437774` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:37.796492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35715f7-06a0-49ef-a66f-cd539bfa5d16 None admin] RBAC: Authorizing `identity:get_project(project_id=a384e760046041c391b684c018437774)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:37.797919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35715f7-06a0-49ef-a66f-cd539bfa5d16 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:37.799655 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 573/1145] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:59:37 2022] GET /identity/v3/projects/a384e760046041c391b684c018437774 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:37.805642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964d048-7927-475c-a019-265d76f40096 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:37.899342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964d048-7927-475c-a019-265d76f40096 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:37.930674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb9576-b971-44a6-af6a-eb5b8f0dcc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:37.931310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb9576-b971-44a6-af6a-eb5b8f0dcc75 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:37.931596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb9576-b971-44a6-af6a-eb5b8f0dcc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:37.931876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb9576-b971-44a6-af6a-eb5b8f0dcc75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:37.932709 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb9576-b971-44a6-af6a-eb5b8f0dcc75 None admin] RBAC: Adding query filter params (role.id=5c87e804d0764309ba08ad6e06b23a87, scope.project.id=a384e760046041c391b684c018437774, user.id=2f8bd0fe63884340b54d21651ddf7313) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:37.933023 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb9576-b971-44a6-af6a-eb5b8f0dcc75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:37.934991 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb9576-b971-44a6-af6a-eb5b8f0dcc75 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:37.940166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 573/1146] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:59:37 2022] GET /identity/v3/role_assignments?user.id=2f8bd0fe63884340b54d21651ddf7313&scope.project.id=a384e760046041c391b684c018437774&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:40.107202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b3742-d5ad-4ada-a122-a7cbaabbff2c None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:40.107725 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b3742-d5ad-4ada-a122-a7cbaabbff2c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:40.107725 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b3742-d5ad-4ada-a122-a7cbaabbff2c None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:40.109094 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 574/1147] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:40.115021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcccdcd-bbbe-433b-a6c9-f12d6d90e336 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:40.115434 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcccdcd-bbbe-433b-a6c9-f12d6d90e336 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:40.115573 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcccdcd-bbbe-433b-a6c9-f12d6d90e336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:40.132004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-5dcccdcd-bbbe-433b-a6c9-f12d6d90e336 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:40.133277 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-5dcccdcd-bbbe-433b-a6c9-f12d6d90e336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:40.195703 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 574/1148] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:59:40.218828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b3742-d5ad-4ada-a122-a7cbaabbff2c None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:40.314898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b3742-d5ad-4ada-a122-a7cbaabbff2c None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:40.346266 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3a28c-202a-453f-8503-0362ca66040f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:40.347035 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3a28c-202a-453f-8503-0362ca66040f None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:40.347317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3a28c-202a-453f-8503-0362ca66040f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:40.347593 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3a28c-202a-453f-8503-0362ca66040f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:40.357827 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3a28c-202a-453f-8503-0362ca66040f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:59:40.365518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-c9e3a28c-202a-453f-8503-0362ca66040f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:59:40.365518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:40.365518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:40.365518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:40.365518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:40.365518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:40.365518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:40.365518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:40.365518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:40.365518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:40.365518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:40.365518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:40.365518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:59:40.365518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:59:40.365518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:59:40.365518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:59:40.366662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:59:40.366662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:59:40.366662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:59:40.366662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:59:40.366662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:40.366662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:40.366662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:40.366662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:40.366662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:40.366662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:40.366662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:40.366662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:59:40.366662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:40.366662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:40.366662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:40.366662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:40.367809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:40.367809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:40.367809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:40.367809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:40.367809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:59:40.367809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:59:40.367809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:59:40.367809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:59:40.367809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:59:40.367809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:59:40.367809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:59:40.367809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:59:40.367809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 575/1149] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:59:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:59:40.373024 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcccdcd-bbbe-433b-a6c9-f12d6d90e336 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:40.466157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcccdcd-bbbe-433b-a6c9-f12d6d90e336 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:40.500002 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e153b-e1d8-49a4-acc3-880a433435b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:40.500658 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e153b-e1d8-49a4-acc3-880a433435b1 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:40.500936 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e153b-e1d8-49a4-acc3-880a433435b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:40.501208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e153b-e1d8-49a4-acc3-880a433435b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:40.501948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e153b-e1d8-49a4-acc3-880a433435b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:40.502277 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e153b-e1d8-49a4-acc3-880a433435b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:40.503621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e153b-e1d8-49a4-acc3-880a433435b1 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:40.511295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 575/1150] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:59:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:40.517285 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3a28c-202a-453f-8503-0362ca66040f None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:40.612378 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3a28c-202a-453f-8503-0362ca66040f None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:40.643194 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2bb73-fb53-44fa-baaa-5f98f5f0bdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:40.644037 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2bb73-fb53-44fa-baaa-5f98f5f0bdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:40.644321 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2bb73-fb53-44fa-baaa-5f98f5f0bdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:40.644597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2bb73-fb53-44fa-baaa-5f98f5f0bdb3 None admin] PATH_INFO: `/v3/users/2f8bd0fe63884340b54d21651ddf7313` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:40.645770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a2bb73-fb53-44fa-baaa-5f98f5f0bdb3 None admin] RBAC: Authorizing `identity:get_user(user_id=2f8bd0fe63884340b54d21651ddf7313)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:40.647237 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a2bb73-fb53-44fa-baaa-5f98f5f0bdb3 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:40.648822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 576/1151] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:59:40 2022] GET /identity/v3/users/2f8bd0fe63884340b54d21651ddf7313 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:40.654826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e153b-e1d8-49a4-acc3-880a433435b1 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:40.750110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e153b-e1d8-49a4-acc3-880a433435b1 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:40.783220 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56685687-d87b-41aa-8694-1e15302e997b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:40.784051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56685687-d87b-41aa-8694-1e15302e997b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:40.784329 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56685687-d87b-41aa-8694-1e15302e997b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:40.784602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56685687-d87b-41aa-8694-1e15302e997b None admin] PATH_INFO: `/v3/projects/a384e760046041c391b684c018437774` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:40.785755 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56685687-d87b-41aa-8694-1e15302e997b None admin] RBAC: Authorizing `identity:get_project(project_id=a384e760046041c391b684c018437774)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:40.787300 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56685687-d87b-41aa-8694-1e15302e997b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:40.788963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 576/1152] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:59:40 2022] GET /identity/v3/projects/a384e760046041c391b684c018437774 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:40.795443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2bb73-fb53-44fa-baaa-5f98f5f0bdb3 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:40.888358 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2bb73-fb53-44fa-baaa-5f98f5f0bdb3 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:40.920125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8beae88-557e-4d3d-8a9f-ab3d76ea5ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:40.920985 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8beae88-557e-4d3d-8a9f-ab3d76ea5ca3 None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:40.921270 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8beae88-557e-4d3d-8a9f-ab3d76ea5ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:40.921545 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8beae88-557e-4d3d-8a9f-ab3d76ea5ca3 None admin] PATH_INFO: `/v3/projects/a384e760046041c391b684c018437774/users/2f8bd0fe63884340b54d21651ddf7313/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:40.923515 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8beae88-557e-4d3d-8a9f-ab3d76ea5ca3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a384e760046041c391b684c018437774, user_id=2f8bd0fe63884340b54d21651ddf7313, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:40.924932 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8beae88-557e-4d3d-8a9f-ab3d76ea5ca3 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:40.984457 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 577/1153] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 19:59:40 2022] PUT /identity/v3/projects/a384e760046041c391b684c018437774/users/2f8bd0fe63884340b54d21651ddf7313/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 19:59:43.164014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2e777-1c65-4b27-8a9b-9b6f2295fea5 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:43.164569 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2e777-1c65-4b27-8a9b-9b6f2295fea5 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:43.164569 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2e777-1c65-4b27-8a9b-9b6f2295fea5 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:43.165892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 577/1154] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:43.171716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03954dc1-6b50-46c1-8ad4-bdeeef1d5156 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:43.172092 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03954dc1-6b50-46c1-8ad4-bdeeef1d5156 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:43.172263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03954dc1-6b50-46c1-8ad4-bdeeef1d5156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:43.188299 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-03954dc1-6b50-46c1-8ad4-bdeeef1d5156 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:43.189462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-03954dc1-6b50-46c1-8ad4-bdeeef1d5156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:43.251798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 578/1155] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:59:43.275439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2e777-1c65-4b27-8a9b-9b6f2295fea5 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:43.373077 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2e777-1c65-4b27-8a9b-9b6f2295fea5 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:43.405316 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0161f-e900-4049-a249-c00ee38bf4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:43.406115 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0161f-e900-4049-a249-c00ee38bf4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:43.406410 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0161f-e900-4049-a249-c00ee38bf4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:43.406688 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0161f-e900-4049-a249-c00ee38bf4a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:43.421811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0161f-e900-4049-a249-c00ee38bf4a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:59:43.429639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-43e0161f-e900-4049-a249-c00ee38bf4a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:59:43.429639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:43.429639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:43.429639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:43.429639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:43.429639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:43.429639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:43.429639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:43.429639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:43.429639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:43.429639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:43.429639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:43.429639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:59:43.429639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:59:43.429639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:59:43.429639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:59:43.430822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:59:43.430822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:59:43.430822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:59:43.430822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:59:43.430822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:43.430822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:43.430822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:43.430822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:43.430822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:43.430822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:43.430822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:43.430822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:59:43.430822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:43.430822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:43.430822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:43.430822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:43.432559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:43.432559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:43.432559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:43.432559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:43.432559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:59:43.432559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:59:43.432559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:59:43.432559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:59:43.432559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:59:43.432559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:59:43.432559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:59:43.432559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:43.432559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 578/1156] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:59:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:59:43.436886 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03954dc1-6b50-46c1-8ad4-bdeeef1d5156 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:43.533461 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03954dc1-6b50-46c1-8ad4-bdeeef1d5156 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:43.564737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2fa242-4d3e-4f5f-968e-ca2c056c78d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:43.565388 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fa242-4d3e-4f5f-968e-ca2c056c78d6 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:43.565673 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fa242-4d3e-4f5f-968e-ca2c056c78d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:43.565950 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fa242-4d3e-4f5f-968e-ca2c056c78d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:43.566721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2fa242-4d3e-4f5f-968e-ca2c056c78d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:43.567029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2fa242-4d3e-4f5f-968e-ca2c056c78d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:43.568392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2fa242-4d3e-4f5f-968e-ca2c056c78d6 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:43.576547 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 579/1157] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:59:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:43.582273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0161f-e900-4049-a249-c00ee38bf4a8 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:43.676949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0161f-e900-4049-a249-c00ee38bf4a8 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:43.709232 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41cdf9-ec44-4999-89f2-169be09c652f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:43.710099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41cdf9-ec44-4999-89f2-169be09c652f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:43.710393 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41cdf9-ec44-4999-89f2-169be09c652f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:43.710670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41cdf9-ec44-4999-89f2-169be09c652f None admin] PATH_INFO: `/v3/users/2f8bd0fe63884340b54d21651ddf7313` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:43.711851 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41cdf9-ec44-4999-89f2-169be09c652f None admin] RBAC: Authorizing `identity:get_user(user_id=2f8bd0fe63884340b54d21651ddf7313)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:43.713272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41cdf9-ec44-4999-89f2-169be09c652f None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:43.714900 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 579/1158] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:59:43 2022] GET /identity/v3/users/2f8bd0fe63884340b54d21651ddf7313 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:43.721727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2fa242-4d3e-4f5f-968e-ca2c056c78d6 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:43.834284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2fa242-4d3e-4f5f-968e-ca2c056c78d6 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:43.865358 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20b96b-ccf5-47a5-b484-4f82eab7ef22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:43.865999 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b96b-ccf5-47a5-b484-4f82eab7ef22 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:43.866314 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b96b-ccf5-47a5-b484-4f82eab7ef22 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:43.866597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b96b-ccf5-47a5-b484-4f82eab7ef22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:43.931113 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20b96b-ccf5-47a5-b484-4f82eab7ef22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:43.932588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20b96b-ccf5-47a5-b484-4f82eab7ef22 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:44.025004 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 580/1159] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:59:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 19:59:44.030873 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41cdf9-ec44-4999-89f2-169be09c652f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:44.129456 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41cdf9-ec44-4999-89f2-169be09c652f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:44.160843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b6dd4-c2d6-458f-be56-35268cf8eee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:44.161681 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b6dd4-c2d6-458f-be56-35268cf8eee5 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:44.161965 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b6dd4-c2d6-458f-be56-35268cf8eee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:44.162286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b6dd4-c2d6-458f-be56-35268cf8eee5 None admin] PATH_INFO: `/v3/projects/a384e760046041c391b684c018437774` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:44.163462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b6dd4-c2d6-458f-be56-35268cf8eee5 None admin] RBAC: Authorizing `identity:get_project(project_id=a384e760046041c391b684c018437774)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:44.164882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b6dd4-c2d6-458f-be56-35268cf8eee5 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:44.166592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 580/1160] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:59:44 2022] GET /identity/v3/projects/a384e760046041c391b684c018437774 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:44.173321 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20b96b-ccf5-47a5-b484-4f82eab7ef22 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:44.267118 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20b96b-ccf5-47a5-b484-4f82eab7ef22 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:44.299007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9b1a6-95bf-4fad-9a52-e660ef8e1e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:44.299633 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9b1a6-95bf-4fad-9a52-e660ef8e1e05 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:44.299911 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9b1a6-95bf-4fad-9a52-e660ef8e1e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:44.300181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9b1a6-95bf-4fad-9a52-e660ef8e1e05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:44.301001 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9b1a6-95bf-4fad-9a52-e660ef8e1e05 None admin] RBAC: Adding query filter params (role.id=5c87e804d0764309ba08ad6e06b23a87, scope.project.id=a384e760046041c391b684c018437774, user.id=2f8bd0fe63884340b54d21651ddf7313) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:44.301304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9b1a6-95bf-4fad-9a52-e660ef8e1e05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:44.302802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9b1a6-95bf-4fad-9a52-e660ef8e1e05 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:44.308333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 581/1161] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:59:44 2022] GET /identity/v3/role_assignments?user.id=2f8bd0fe63884340b54d21651ddf7313&scope.project.id=a384e760046041c391b684c018437774&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:46.506570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2305254-caa6-4ac5-9f71-505269b13272 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:46.507128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2305254-caa6-4ac5-9f71-505269b13272 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:46.507128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2305254-caa6-4ac5-9f71-505269b13272 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:46.508473 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 581/1162] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:46.514385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c2ef1-9539-4aaf-bd56-b9ff45794301 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:46.514813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c2ef1-9539-4aaf-bd56-b9ff45794301 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:46.514946 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c2ef1-9539-4aaf-bd56-b9ff45794301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:46.531745 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-b29c2ef1-9539-4aaf-bd56-b9ff45794301 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:46.532919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b29c2ef1-9539-4aaf-bd56-b9ff45794301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:46.595074 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 582/1163] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:59:46.617154 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2305254-caa6-4ac5-9f71-505269b13272 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:46.713230 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2305254-caa6-4ac5-9f71-505269b13272 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:46.744470 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084835b-b0ea-4efa-8385-97e892fe60f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:46.745097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084835b-b0ea-4efa-8385-97e892fe60f8 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:46.745377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084835b-b0ea-4efa-8385-97e892fe60f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:46.745647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084835b-b0ea-4efa-8385-97e892fe60f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:46.807648 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084835b-b0ea-4efa-8385-97e892fe60f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:46.809601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084835b-b0ea-4efa-8385-97e892fe60f8 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:46.901086 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 582/1164] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:59:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 19:59:46.907264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c2ef1-9539-4aaf-bd56-b9ff45794301 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:46.999834 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c2ef1-9539-4aaf-bd56-b9ff45794301 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:47.031149 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66001308-bdce-42b4-91df-22849510befe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:47.031936 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66001308-bdce-42b4-91df-22849510befe None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:47.032222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66001308-bdce-42b4-91df-22849510befe None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:47.032500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66001308-bdce-42b4-91df-22849510befe None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:47.038944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66001308-bdce-42b4-91df-22849510befe None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:47.040369 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66001308-bdce-42b4-91df-22849510befe None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:47.054697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-66001308-bdce-42b4-91df-22849510befe None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 10 19:59:47.054697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:47.054697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 10 19:59:47.054697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 10 19:59:47.054697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 19:59:47.054697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 19:59:47.054697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 19:59:47.054697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 19:59:47.054697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 10 19:59:47.054697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:59:47.054697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 19:59:47.054697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:59:47.054697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:47.054697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:47.054697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:47.054697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:47.054697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:47.055973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:47.055973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:47.055973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:47.055973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:47.055973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:47.055973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:47.055973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 19:59:47.055973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 19:59:47.055973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 19:59:47.055973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 19:59:47.055973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:47.055973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:47.055973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:47.055973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:47.055973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:47.055973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:47.057144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:47.057144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 19:59:47.057144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:47.057144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:47.057144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:47.057144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:47.057144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:47.057144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:47.057144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:47.057144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:47.057144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 10 19:59:47.057144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 19:59:47.057144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 10 19:59:47.057144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 19:59:47.057144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 583/1165] 10.222.0.65 () {56 vars in 1117 bytes} [Tue May 10 19:59:46 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:59:47.062289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084835b-b0ea-4efa-8385-97e892fe60f8 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:47.160756 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084835b-b0ea-4efa-8385-97e892fe60f8 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:47.193401 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb7a19-7db3-40b1-b886-864217afdeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:47.194050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb7a19-7db3-40b1-b886-864217afdeea None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:47.194364 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb7a19-7db3-40b1-b886-864217afdeea None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:47.194642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb7a19-7db3-40b1-b886-864217afdeea None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:47.195363 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb7a19-7db3-40b1-b886-864217afdeea None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:47.195671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb7a19-7db3-40b1-b886-864217afdeea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:47.197035 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb7a19-7db3-40b1-b886-864217afdeea None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:47.203931 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 583/1166] 10.222.0.65 () {56 vars in 1115 bytes} [Tue May 10 19:59:47 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:49.382820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33baf02-4ccd-483f-9ff3-165a40af1ada None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:49.383374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33baf02-4ccd-483f-9ff3-165a40af1ada None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:49.383374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33baf02-4ccd-483f-9ff3-165a40af1ada None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:49.384742 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 584/1167] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:49.390491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099163b-dc5e-4f8c-9081-6ec586e71d6d None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:49.390923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099163b-dc5e-4f8c-9081-6ec586e71d6d None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:49.391050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099163b-dc5e-4f8c-9081-6ec586e71d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:49.407796 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-a099163b-dc5e-4f8c-9081-6ec586e71d6d None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:49.408983 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a099163b-dc5e-4f8c-9081-6ec586e71d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:49.471687 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 584/1168] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:59:49.493837 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33baf02-4ccd-483f-9ff3-165a40af1ada None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:49.590515 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33baf02-4ccd-483f-9ff3-165a40af1ada None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:49.622092 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8163c8f-1bed-431e-8a65-b6305b47fc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:49.622787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8163c8f-1bed-431e-8a65-b6305b47fc7b None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:49.623073 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8163c8f-1bed-431e-8a65-b6305b47fc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:49.623353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8163c8f-1bed-431e-8a65-b6305b47fc7b None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:49.624107 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8163c8f-1bed-431e-8a65-b6305b47fc7b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:49.625471 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8163c8f-1bed-431e-8a65-b6305b47fc7b None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:49.682069 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 585/1169] 10.222.0.65 () {60 vars in 1138 bytes} [Tue May 10 19:59:49 2022] POST /identity/v3/domains => generated 267 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:59:51.882377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3fb39-d253-4106-b935-5d03b66b4423 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:51.882933 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3fb39-d253-4106-b935-5d03b66b4423 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:51.882933 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3fb39-d253-4106-b935-5d03b66b4423 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:51.884290 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 585/1170] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:51.890134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9ae34-a7f9-4c20-a34e-522485039e2b None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:51.890549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9ae34-a7f9-4c20-a34e-522485039e2b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:51.890702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9ae34-a7f9-4c20-a34e-522485039e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:51.907035 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-37c9ae34-a7f9-4c20-a34e-522485039e2b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:51.908213 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-37c9ae34-a7f9-4c20-a34e-522485039e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:51.971161 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 586/1171] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:59:51.993804 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3fb39-d253-4106-b935-5d03b66b4423 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:52.088957 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3fb39-d253-4106-b935-5d03b66b4423 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:52.120376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea160c7-0381-4202-8980-288b814519ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:52.121711 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea160c7-0381-4202-8980-288b814519ec None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:52.121991 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea160c7-0381-4202-8980-288b814519ec None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:52.122291 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea160c7-0381-4202-8980-288b814519ec None admin] PATH_INFO: `/v3/domains/5d9a3517cf6a49cd862fa2c5b952646d` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:52.132152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea160c7-0381-4202-8980-288b814519ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=5d9a3517cf6a49cd862fa2c5b952646d)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:52.133638 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea160c7-0381-4202-8980-288b814519ec None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:52.135215 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 586/1172] 10.222.0.65 () {56 vars in 1183 bytes} [Tue May 10 19:59:51 2022] GET /identity/v3/domains/5d9a3517cf6a49cd862fa2c5b952646d => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:52.141323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9ae34-a7f9-4c20-a34e-522485039e2b None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:52.233080 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9ae34-a7f9-4c20-a34e-522485039e2b None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:52.264789 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff998a-52b4-4ea7-b395-376c43088061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:52.265424 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff998a-52b4-4ea7-b395-376c43088061 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:52.265709 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff998a-52b4-4ea7-b395-376c43088061 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:52.265982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff998a-52b4-4ea7-b395-376c43088061 None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:52.266714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff998a-52b4-4ea7-b395-376c43088061 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:52.268116 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff998a-52b4-4ea7-b395-376c43088061 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:52.367981 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 587/1173] 10.222.0.65 () {60 vars in 1141 bytes} [Tue May 10 19:59:52 2022] POST /identity/v3/projects => generated 361 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:59:54.595227 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25c847-178c-4d7a-b9bf-39f06e1be5c4 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:54.595765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25c847-178c-4d7a-b9bf-39f06e1be5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:54.595765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25c847-178c-4d7a-b9bf-39f06e1be5c4 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:54.597126 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 587/1174] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:54.602280 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba75800-9363-4c9e-a449-ed2553775e71 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:54.602701 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba75800-9363-4c9e-a449-ed2553775e71 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:54.602839 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba75800-9363-4c9e-a449-ed2553775e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:54.619927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-5ba75800-9363-4c9e-a449-ed2553775e71 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:54.621097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-5ba75800-9363-4c9e-a449-ed2553775e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:54.684064 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 588/1175] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 19:59:54.708292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25c847-178c-4d7a-b9bf-39f06e1be5c4 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:54.804334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25c847-178c-4d7a-b9bf-39f06e1be5c4 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:54.835531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc60b05-3cee-4def-b805-6bf9385c3e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:54.836374 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc60b05-3cee-4def-b805-6bf9385c3e8d None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:54.836659 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc60b05-3cee-4def-b805-6bf9385c3e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:54.836938 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc60b05-3cee-4def-b805-6bf9385c3e8d None admin] PATH_INFO: `/v3/domains/5d9a3517cf6a49cd862fa2c5b952646d` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:54.838127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc60b05-3cee-4def-b805-6bf9385c3e8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=5d9a3517cf6a49cd862fa2c5b952646d)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:54.840155 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc60b05-3cee-4def-b805-6bf9385c3e8d None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:54.841760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 588/1176] 10.222.0.65 () {56 vars in 1183 bytes} [Tue May 10 19:59:54 2022] GET /identity/v3/domains/5d9a3517cf6a49cd862fa2c5b952646d => generated 267 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 19:59:54.847350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba75800-9363-4c9e-a449-ed2553775e71 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:54.942856 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba75800-9363-4c9e-a449-ed2553775e71 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:54.974726 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55b5da-992a-40c6-b081-e30414678495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:54.975376 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55b5da-992a-40c6-b081-e30414678495 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:54.975654 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55b5da-992a-40c6-b081-e30414678495 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:54.975925 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55b5da-992a-40c6-b081-e30414678495 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:54.976665 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55b5da-992a-40c6-b081-e30414678495 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:54.978075 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55b5da-992a-40c6-b081-e30414678495 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:55.092134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 589/1177] 10.222.0.65 () {60 vars in 1132 bytes} [Tue May 10 19:59:54 2022] POST /identity/v3/users => generated 311 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 19:59:57.363028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643b0b2-423b-4b6b-8e8f-6e3ec30441ed None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:57.363565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643b0b2-423b-4b6b-8e8f-6e3ec30441ed None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:57.363565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643b0b2-423b-4b6b-8e8f-6e3ec30441ed None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:57.364932 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 589/1178] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 19:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 19:59:57.370214 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831d298-d143-462e-b2c8-c65d6897df8d None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:57.371133 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831d298-d143-462e-b2c8-c65d6897df8d None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:57.371319 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831d298-d143-462e-b2c8-c65d6897df8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:57.388182 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-a831d298-d143-462e-b2c8-c65d6897df8d None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 19:59:57.389360 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a831d298-d143-462e-b2c8-c65d6897df8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 19:59:57.452532 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 590/1179] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 19:59:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 19:59:57.475254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643b0b2-423b-4b6b-8e8f-6e3ec30441ed None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:57.572253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643b0b2-423b-4b6b-8e8f-6e3ec30441ed None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:57.603190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8504b8-8415-4d6c-9b60-1d8419651906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:57.603972 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8504b8-8415-4d6c-9b60-1d8419651906 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:57.604258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8504b8-8415-4d6c-9b60-1d8419651906 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:57.604535 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8504b8-8415-4d6c-9b60-1d8419651906 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:57.615467 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8504b8-8415-4d6c-9b60-1d8419651906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 19:59:57.623543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-4e8504b8-8415-4d6c-9b60-1d8419651906 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:59:57.623543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 19:59:57.623543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 19:59:57.623543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 19:59:57.623543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 19:59:57.623543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 19:59:57.623543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 19:59:57.623543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 19:59:57.623543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 19:59:57.623543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 19:59:57.623543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 19:59:57.623543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 19:59:57.623543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 19:59:57.623543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 19:59:57.623543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 19:59:57.623543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 19:59:57.624682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 19:59:57.624682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 19:59:57.624682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 19:59:57.624682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 19:59:57.624682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 19:59:57.624682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 19:59:57.624682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 19:59:57.624682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 19:59:57.624682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 19:59:57.624682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 19:59:57.624682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 19:59:57.624682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 19:59:57.624682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 19:59:57.624682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 19:59:57.624682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 19:59:57.624682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 19:59:57.625763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 19:59:57.625763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 19:59:57.625763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 19:59:57.625763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 19:59:57.625763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 19:59:57.625763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 19:59:57.625763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 19:59:57.625763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 19:59:57.625763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 19:59:57.625763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 19:59:57.625763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 19:59:57.625763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 19:59:57.625763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 590/1180] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 19:59:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 19:59:57.630815 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831d298-d143-462e-b2c8-c65d6897df8d None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:57.722956 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831d298-d143-462e-b2c8-c65d6897df8d None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:57.755404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf55d2-0255-4acb-ad0f-a15dfd1e5b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:57.756052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf55d2-0255-4acb-ad0f-a15dfd1e5b03 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:57.756334 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf55d2-0255-4acb-ad0f-a15dfd1e5b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:57.756609 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf55d2-0255-4acb-ad0f-a15dfd1e5b03 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:57.757348 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf55d2-0255-4acb-ad0f-a15dfd1e5b03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:57.757653 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf55d2-0255-4acb-ad0f-a15dfd1e5b03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:57.759024 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf55d2-0255-4acb-ad0f-a15dfd1e5b03 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:57.766646 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 591/1181] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 19:59:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:57.772221 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8504b8-8415-4d6c-9b60-1d8419651906 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:57.889304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8504b8-8415-4d6c-9b60-1d8419651906 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:57.921874 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e6ebc-375b-4511-9684-c0a148c68d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:57.922750 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e6ebc-375b-4511-9684-c0a148c68d05 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:57.923030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e6ebc-375b-4511-9684-c0a148c68d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:57.923300 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e6ebc-375b-4511-9684-c0a148c68d05 None admin] PATH_INFO: `/v3/users/b5f26571089c49c7a5a33ad505e7ffb1` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:57.935413 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e6ebc-375b-4511-9684-c0a148c68d05 None admin] RBAC: Authorizing `identity:get_user(user_id=b5f26571089c49c7a5a33ad505e7ffb1)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:57.936833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e6ebc-375b-4511-9684-c0a148c68d05 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:57.938481 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 591/1182] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 19:59:57 2022] GET /identity/v3/users/b5f26571089c49c7a5a33ad505e7ffb1 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:57.944176 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf55d2-0255-4acb-ad0f-a15dfd1e5b03 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:58.037447 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf55d2-0255-4acb-ad0f-a15dfd1e5b03 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:58.068549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44be491-fcdf-4a3f-a461-cbd36301a71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:58.069170 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44be491-fcdf-4a3f-a461-cbd36301a71a None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:58.069450 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44be491-fcdf-4a3f-a461-cbd36301a71a None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:58.069723 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44be491-fcdf-4a3f-a461-cbd36301a71a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:58.132478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44be491-fcdf-4a3f-a461-cbd36301a71a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:58.133908 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44be491-fcdf-4a3f-a461-cbd36301a71a None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:58.231948 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 592/1183] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 19:59:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 19:59:58.238122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e6ebc-375b-4511-9684-c0a148c68d05 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:58.330839 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e6ebc-375b-4511-9684-c0a148c68d05 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:58.363537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbccfa6b-ca8a-4f71-9257-5529c3bacb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:58.364368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccfa6b-ca8a-4f71-9257-5529c3bacb15 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:58.364648 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccfa6b-ca8a-4f71-9257-5529c3bacb15 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:58.364920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccfa6b-ca8a-4f71-9257-5529c3bacb15 None admin] PATH_INFO: `/v3/projects/5a52048a6bd94e21ae56e33c8e273098` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:58.366116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbccfa6b-ca8a-4f71-9257-5529c3bacb15 None admin] RBAC: Authorizing `identity:get_project(project_id=5a52048a6bd94e21ae56e33c8e273098)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:58.367537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbccfa6b-ca8a-4f71-9257-5529c3bacb15 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:58.369190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 592/1184] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 19:59:58 2022] GET /identity/v3/projects/5a52048a6bd94e21ae56e33c8e273098 => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 19:59:58.375104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44be491-fcdf-4a3f-a461-cbd36301a71a None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 19:59:58.467362 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44be491-fcdf-4a3f-a461-cbd36301a71a None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 19:59:58.501172 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ad18e-44b3-47e3-a729-0a746a634b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 19:59:58.501745 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ad18e-44b3-47e3-a729-0a746a634b21 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 19:59:58.502027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ad18e-44b3-47e3-a729-0a746a634b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 19:59:58.502330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ad18e-44b3-47e3-a729-0a746a634b21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 19:59:58.503160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ad18e-44b3-47e3-a729-0a746a634b21 None admin] RBAC: Adding query filter params (role.id=5c87e804d0764309ba08ad6e06b23a87, scope.project.id=5a52048a6bd94e21ae56e33c8e273098, user.id=b5f26571089c49c7a5a33ad505e7ffb1) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 19:59:58.503470 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ad18e-44b3-47e3-a729-0a746a634b21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 19:59:58.504895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ad18e-44b3-47e3-a729-0a746a634b21 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 19:59:58.510127 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 593/1185] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 19:59:58 2022] GET /identity/v3/role_assignments?user.id=b5f26571089c49c7a5a33ad505e7ffb1&scope.project.id=5a52048a6bd94e21ae56e33c8e273098&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:00:00.689124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7df53-da3b-48d0-91e2-85477cc83abc None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:00:00.689649 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7df53-da3b-48d0-91e2-85477cc83abc None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:00:00.689764 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7df53-da3b-48d0-91e2-85477cc83abc None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:00:00.691071 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 593/1186] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:00:00.696168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0cd82-2f69-4a71-9347-04664ac915f3 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:00:00.696556 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0cd82-2f69-4a71-9347-04664ac915f3 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:00:00.696721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0cd82-2f69-4a71-9347-04664ac915f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:00:00.712960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-00e0cd82-2f69-4a71-9347-04664ac915f3 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:00:00.714161 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-00e0cd82-2f69-4a71-9347-04664ac915f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:00:00.777117 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 594/1187] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:00:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 20:00:00.799309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7df53-da3b-48d0-91e2-85477cc83abc None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:00:00.894734 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7df53-da3b-48d0-91e2-85477cc83abc None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:00:00.926152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124a167-0fa8-4b7a-bdd9-0f0e8c529316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:00:00.926929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124a167-0fa8-4b7a-bdd9-0f0e8c529316 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:00:00.927215 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124a167-0fa8-4b7a-bdd9-0f0e8c529316 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:00:00.927496 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124a167-0fa8-4b7a-bdd9-0f0e8c529316 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:00:00.937602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124a167-0fa8-4b7a-bdd9-0f0e8c529316 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 20:00:00.945194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-a124a167-0fa8-4b7a-bdd9-0f0e8c529316 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 20:00:00.945194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:00:00.945194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:00:00.945194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:00:00.945194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:00:00.945194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:00:00.945194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:00:00.945194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:00:00.945194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:00:00.945194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:00:00.945194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:00:00.945194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:00:00.945194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 20:00:00.945194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 20:00:00.945194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 20:00:00.945194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 20:00:00.946334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 20:00:00.946334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 20:00:00.946334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 20:00:00.946334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 20:00:00.946334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:00:00.946334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:00:00.946334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:00:00.946334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:00:00.946334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:00:00.946334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:00:00.946334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:00:00.946334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:00:00.946334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:00:00.946334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:00:00.946334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:00:00.946334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:00:00.947379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:00:00.947379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:00:00.947379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:00:00.947379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:00:00.947379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 20:00:00.947379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 20:00:00.947379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 20:00:00.947379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 20:00:00.947379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 20:00:00.947379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 20:00:00.947379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 20:00:00.947379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:00:00.947379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 594/1188] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 20:00:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:00:00.952616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0cd82-2f69-4a71-9347-04664ac915f3 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:00:01.044370 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0cd82-2f69-4a71-9347-04664ac915f3 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:00:01.076524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895d2af-a938-45c4-b52e-7ad02b0f3eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:00:01.077175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895d2af-a938-45c4-b52e-7ad02b0f3eaa None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:00:01.077458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895d2af-a938-45c4-b52e-7ad02b0f3eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:00:01.077736 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895d2af-a938-45c4-b52e-7ad02b0f3eaa None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:00:01.078508 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895d2af-a938-45c4-b52e-7ad02b0f3eaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:00:01.078814 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895d2af-a938-45c4-b52e-7ad02b0f3eaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:00:01.080164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895d2af-a938-45c4-b52e-7ad02b0f3eaa None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:00:01.087780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 595/1189] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 20:00:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:00:01.093580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124a167-0fa8-4b7a-bdd9-0f0e8c529316 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:00:01.188190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124a167-0fa8-4b7a-bdd9-0f0e8c529316 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:00:01.220256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa613b17-b93c-4a04-89d0-efce33f6f42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:00:01.221037 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa613b17-b93c-4a04-89d0-efce33f6f42e None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:00:01.221320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa613b17-b93c-4a04-89d0-efce33f6f42e None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:00:01.221596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa613b17-b93c-4a04-89d0-efce33f6f42e None admin] PATH_INFO: `/v3/users/b5f26571089c49c7a5a33ad505e7ffb1` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:00:01.222788 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa613b17-b93c-4a04-89d0-efce33f6f42e None admin] RBAC: Authorizing `identity:get_user(user_id=b5f26571089c49c7a5a33ad505e7ffb1)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:00:01.224202 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa613b17-b93c-4a04-89d0-efce33f6f42e None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:00:01.225783 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 595/1190] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 20:00:01 2022] GET /identity/v3/users/b5f26571089c49c7a5a33ad505e7ffb1 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:00:01.231832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895d2af-a938-45c4-b52e-7ad02b0f3eaa None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:00:01.326391 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895d2af-a938-45c4-b52e-7ad02b0f3eaa None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:00:01.358912 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b2af7-4ac9-4853-95db-77ff12b9f682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:00:01.359751 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b2af7-4ac9-4853-95db-77ff12b9f682 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:00:01.360032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b2af7-4ac9-4853-95db-77ff12b9f682 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:00:01.360306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b2af7-4ac9-4853-95db-77ff12b9f682 None admin] PATH_INFO: `/v3/projects/5a52048a6bd94e21ae56e33c8e273098` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:00:01.361484 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b2af7-4ac9-4853-95db-77ff12b9f682 None admin] RBAC: Authorizing `identity:get_project(project_id=5a52048a6bd94e21ae56e33c8e273098)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:00:01.362946 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b2af7-4ac9-4853-95db-77ff12b9f682 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:00:01.364630 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 596/1191] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 20:00:01 2022] GET /identity/v3/projects/5a52048a6bd94e21ae56e33c8e273098 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:00:01.370398 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa613b17-b93c-4a04-89d0-efce33f6f42e None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:00:01.464641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa613b17-b93c-4a04-89d0-efce33f6f42e None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:00:01.495738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eaa63f-f416-4f62-815b-aa42bb4125a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:00:01.496555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eaa63f-f416-4f62-815b-aa42bb4125a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:00:01.496840 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eaa63f-f416-4f62-815b-aa42bb4125a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:00:01.497118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eaa63f-f416-4f62-815b-aa42bb4125a8 None admin] PATH_INFO: `/v3/projects/5a52048a6bd94e21ae56e33c8e273098/users/b5f26571089c49c7a5a33ad505e7ffb1/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:00:01.499099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eaa63f-f416-4f62-815b-aa42bb4125a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a52048a6bd94e21ae56e33c8e273098, user_id=b5f26571089c49c7a5a33ad505e7ffb1, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:00:01.500521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eaa63f-f416-4f62-815b-aa42bb4125a8 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:00:01.566024 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 596/1192] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 20:00:01 2022] PUT /identity/v3/projects/5a52048a6bd94e21ae56e33c8e273098/users/b5f26571089c49c7a5a33ad505e7ffb1/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:00:03.755482 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60947adf-a55c-4214-a44f-c95c94f1d27a None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:00:03.756097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60947adf-a55c-4214-a44f-c95c94f1d27a None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:00:03.756097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60947adf-a55c-4214-a44f-c95c94f1d27a None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:00:03.757404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 597/1193] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:00:03.763377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2467ec-6ff0-4364-9f58-dd1ad728f444 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:00:03.763806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2467ec-6ff0-4364-9f58-dd1ad728f444 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:00:03.763943 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2467ec-6ff0-4364-9f58-dd1ad728f444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:00:03.780835 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-bb2467ec-6ff0-4364-9f58-dd1ad728f444 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:00:03.782016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-bb2467ec-6ff0-4364-9f58-dd1ad728f444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:00:03.845124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 597/1194] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:00:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 20:00:03.868129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60947adf-a55c-4214-a44f-c95c94f1d27a None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:00:03.965947 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60947adf-a55c-4214-a44f-c95c94f1d27a None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:00:03.997682 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc53ce-683a-4eb2-b537-5fd8dd01daa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:00:03.998490 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc53ce-683a-4eb2-b537-5fd8dd01daa2 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:00:03.998776 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc53ce-683a-4eb2-b537-5fd8dd01daa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:00:03.999053 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc53ce-683a-4eb2-b537-5fd8dd01daa2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:00:04.009114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc53ce-683a-4eb2-b537-5fd8dd01daa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 20:00:04.017183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-b1dc53ce-683a-4eb2-b537-5fd8dd01daa2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 20:00:04.017183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:00:04.017183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:00:04.017183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:00:04.017183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:00:04.017183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:00:04.017183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:00:04.017183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:00:04.017183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:00:04.017183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:00:04.017183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:00:04.017183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:00:04.017183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 20:00:04.017183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 20:00:04.017183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 20:00:04.017183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 20:00:04.018402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 20:00:04.018402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 20:00:04.018402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 20:00:04.018402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 20:00:04.018402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:00:04.018402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:00:04.018402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:00:04.018402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:00:04.018402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:00:04.018402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:00:04.018402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:00:04.018402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 20:00:04.018402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:00:04.018402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 20:00:04.018402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:00:04.018402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:00:04.019651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:00:04.019651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 20:00:04.019651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:00:04.019651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 20:00:04.019651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 20:00:04.019651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 20:00:04.019651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 20:00:04.019651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 20:00:04.019651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 20:00:04.019651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 20:00:04.019651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 20:00:04.019651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 20:00:04.019651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 598/1195] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 20:00:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:00:04.024429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2467ec-6ff0-4364-9f58-dd1ad728f444 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:00:04.117689 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2467ec-6ff0-4364-9f58-dd1ad728f444 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:00:04.150050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc1f1f-5de0-4a8b-8332-921ed6dbf9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:00:04.150727 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc1f1f-5de0-4a8b-8332-921ed6dbf9fc None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:00:04.151009 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc1f1f-5de0-4a8b-8332-921ed6dbf9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:00:04.151324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc1f1f-5de0-4a8b-8332-921ed6dbf9fc None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:00:04.152071 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc1f1f-5de0-4a8b-8332-921ed6dbf9fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:00:04.152375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc1f1f-5de0-4a8b-8332-921ed6dbf9fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:00:04.153745 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc1f1f-5de0-4a8b-8332-921ed6dbf9fc None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:00:04.161424 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 598/1196] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 20:00:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:00:04.167063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc53ce-683a-4eb2-b537-5fd8dd01daa2 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:00:04.265706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc53ce-683a-4eb2-b537-5fd8dd01daa2 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:00:04.294385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf6516-6f7f-45ca-ab7c-18d219d8fe9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:00:04.295246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf6516-6f7f-45ca-ab7c-18d219d8fe9d None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:00:04.295523 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf6516-6f7f-45ca-ab7c-18d219d8fe9d None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:00:04.295790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf6516-6f7f-45ca-ab7c-18d219d8fe9d None admin] PATH_INFO: `/v3/users/b5f26571089c49c7a5a33ad505e7ffb1` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:00:04.296996 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf6516-6f7f-45ca-ab7c-18d219d8fe9d None admin] RBAC: Authorizing `identity:get_user(user_id=b5f26571089c49c7a5a33ad505e7ffb1)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:00:04.298432 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf6516-6f7f-45ca-ab7c-18d219d8fe9d None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:00:04.300057 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 599/1197] 10.222.0.65 () {56 vars in 1177 bytes} [Tue May 10 20:00:04 2022] GET /identity/v3/users/b5f26571089c49c7a5a33ad505e7ffb1 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:00:04.305827 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc1f1f-5de0-4a8b-8332-921ed6dbf9fc None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:00:04.400111 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc1f1f-5de0-4a8b-8332-921ed6dbf9fc None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:00:04.431526 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1853ea8-c74c-4c7e-8c4f-44d2b2107ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:00:04.432160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1853ea8-c74c-4c7e-8c4f-44d2b2107ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:00:04.432440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1853ea8-c74c-4c7e-8c4f-44d2b2107ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:00:04.433343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1853ea8-c74c-4c7e-8c4f-44d2b2107ec9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:00:04.495711 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1853ea8-c74c-4c7e-8c4f-44d2b2107ec9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:00:04.497126 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1853ea8-c74c-4c7e-8c4f-44d2b2107ec9 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:00:04.588622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 599/1198] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 20:00:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 20:00:04.595199 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf6516-6f7f-45ca-ab7c-18d219d8fe9d None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:00:04.688674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf6516-6f7f-45ca-ab7c-18d219d8fe9d None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:00:04.720489 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95248336-5621-475b-ae52-d203e10249a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:00:04.721319 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95248336-5621-475b-ae52-d203e10249a1 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:00:04.721598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95248336-5621-475b-ae52-d203e10249a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:00:04.721870 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95248336-5621-475b-ae52-d203e10249a1 None admin] PATH_INFO: `/v3/projects/5a52048a6bd94e21ae56e33c8e273098` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:00:04.723078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95248336-5621-475b-ae52-d203e10249a1 None admin] RBAC: Authorizing `identity:get_project(project_id=5a52048a6bd94e21ae56e33c8e273098)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:00:04.724495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95248336-5621-475b-ae52-d203e10249a1 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:00:04.726191 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 600/1199] 10.222.0.65 () {56 vars in 1186 bytes} [Tue May 10 20:00:04 2022] GET /identity/v3/projects/5a52048a6bd94e21ae56e33c8e273098 => generated 361 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:00:04.732562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1853ea8-c74c-4c7e-8c4f-44d2b2107ec9 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:00:04.824797 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1853ea8-c74c-4c7e-8c4f-44d2b2107ec9 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:00:04.855842 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb26d0-1cd5-4baf-a64c-dd46829a97f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:00:04.856419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb26d0-1cd5-4baf-a64c-dd46829a97f1 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:00:04.856700 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb26d0-1cd5-4baf-a64c-dd46829a97f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:00:04.856973 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb26d0-1cd5-4baf-a64c-dd46829a97f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:00:04.857798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb26d0-1cd5-4baf-a64c-dd46829a97f1 None admin] RBAC: Adding query filter params (role.id=5c87e804d0764309ba08ad6e06b23a87, scope.project.id=5a52048a6bd94e21ae56e33c8e273098, user.id=b5f26571089c49c7a5a33ad505e7ffb1) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:00:04.858378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb26d0-1cd5-4baf-a64c-dd46829a97f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:00:04.859789 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb26d0-1cd5-4baf-a64c-dd46829a97f1 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:00:04.865312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 600/1200] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 20:00:04 2022] GET /identity/v3/role_assignments?user.id=b5f26571089c49c7a5a33ad505e7ffb1&scope.project.id=5a52048a6bd94e21ae56e33c8e273098&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 608 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:19.650540 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92499168-7321-40a2-9ac5-1cb4d2642991 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:19.651124 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92499168-7321-40a2-9ac5-1cb4d2642991 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:19.651124 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92499168-7321-40a2-9ac5-1cb4d2642991 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:19.652496 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 601/1201] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:01:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:01:19.657688 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ca883-ac31-4aab-97d4-6a6c34cbe299 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:19.658195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ca883-ac31-4aab-97d4-6a6c34cbe299 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:19.658291 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ca883-ac31-4aab-97d4-6a6c34cbe299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:19.675483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-246ca883-ac31-4aab-97d4-6a6c34cbe299 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:01:19.676666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-246ca883-ac31-4aab-97d4-6a6c34cbe299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:01:19.741563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 601/1202] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:01:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 20:01:19.765570 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92499168-7321-40a2-9ac5-1cb4d2642991 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:19.864209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92499168-7321-40a2-9ac5-1cb4d2642991 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:19.898102 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6db7c-c47b-48bc-9667-7c02f8bc9d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:19.898932 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6db7c-c47b-48bc-9667-7c02f8bc9d0b None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:19.899772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6db7c-c47b-48bc-9667-7c02f8bc9d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:19.900050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6db7c-c47b-48bc-9667-7c02f8bc9d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:19.911038 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6db7c-c47b-48bc-9667-7c02f8bc9d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:19.912819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6db7c-c47b-48bc-9667-7c02f8bc9d0b None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:19.934129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-ecf6db7c-c47b-48bc-9667-7c02f8bc9d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:19.934129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:19.934129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:19.934129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:19.934129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:19.934129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:19.934129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:19.934129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:19.934129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:19.934129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:19.934129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:19.934129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:19.934129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 20:01:19.934129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 20:01:19.934129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 20:01:19.935772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 20:01:19.935772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:19.935772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:19.935772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:19.935772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:19.935772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:19.935772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:19.935772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:19.935772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 20:01:19.935772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:19.935772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:19.935772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:19.935772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:19.935772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:19.935772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:19.935772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:19.935772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:19.937598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 20:01:19.937598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 20:01:19.937598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:19.937598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 20:01:19.937598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 602/1203] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:19.941337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ca883-ac31-4aab-97d4-6a6c34cbe299 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:20.036310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ca883-ac31-4aab-97d4-6a6c34cbe299 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:20.067431 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd55304-3d82-4d85-a532-abe1af42331c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:20.068076 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd55304-3d82-4d85-a532-abe1af42331c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:20.068358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd55304-3d82-4d85-a532-abe1af42331c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:20.068629 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd55304-3d82-4d85-a532-abe1af42331c None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:20.069350 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd55304-3d82-4d85-a532-abe1af42331c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:20.069653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd55304-3d82-4d85-a532-abe1af42331c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:20.071557 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd55304-3d82-4d85-a532-abe1af42331c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:20.082273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 602/1204] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 20:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:20.088548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6db7c-c47b-48bc-9667-7c02f8bc9d0b None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:20.190241 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6db7c-c47b-48bc-9667-7c02f8bc9d0b None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:20.234195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67b35c-17f4-43a3-bb44-c2644e268bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:20.237315 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67b35c-17f4-43a3-bb44-c2644e268bdb None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:20.237553 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67b35c-17f4-43a3-bb44-c2644e268bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:20.237836 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67b35c-17f4-43a3-bb44-c2644e268bdb None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:20.244113 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67b35c-17f4-43a3-bb44-c2644e268bdb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:20.245554 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67b35c-17f4-43a3-bb44-c2644e268bdb None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:20.339214 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 603/1205] 10.222.0.65 () {60 vars in 1132 bytes} [Tue May 10 20:01:20 2022] POST /identity/v3/users => generated 251 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:01:22.570537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d0f91-73a9-4f2f-ae73-9d34b1153af1 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:22.571087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d0f91-73a9-4f2f-ae73-9d34b1153af1 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:22.571087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d0f91-73a9-4f2f-ae73-9d34b1153af1 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:22.572641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 603/1206] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:01:22.577960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0df998-a692-4ab2-a556-9b1a6fca1f9a None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:22.578377 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0df998-a692-4ab2-a556-9b1a6fca1f9a None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:22.578536 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0df998-a692-4ab2-a556-9b1a6fca1f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:22.595223 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-cd0df998-a692-4ab2-a556-9b1a6fca1f9a None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:01:22.596396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-cd0df998-a692-4ab2-a556-9b1a6fca1f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:01:22.661115 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 604/1207] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:01:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 20:01:22.684329 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d0f91-73a9-4f2f-ae73-9d34b1153af1 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:22.782239 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d0f91-73a9-4f2f-ae73-9d34b1153af1 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:22.814286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f8bd5-376b-4bc3-94bf-413a88dc87e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:22.815072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f8bd5-376b-4bc3-94bf-413a88dc87e9 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:22.815357 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f8bd5-376b-4bc3-94bf-413a88dc87e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:22.815633 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f8bd5-376b-4bc3-94bf-413a88dc87e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:22.826468 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f8bd5-376b-4bc3-94bf-413a88dc87e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 20:01:22.833992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-f14f8bd5-376b-4bc3-94bf-413a88dc87e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 20:01:22.833992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:22.833992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:22.833992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:22.833992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:22.833992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:22.833992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:22.833992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:22.833992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:22.833992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:22.833992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:22.833992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:22.833992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 20:01:22.833992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 20:01:22.833992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 20:01:22.833992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 20:01:22.835203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 20:01:22.835203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 20:01:22.835203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 20:01:22.835203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 20:01:22.835203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:22.835203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:22.835203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:22.835203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:22.835203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:22.835203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:22.835203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:22.835203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:01:22.835203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:22.835203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:22.835203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:22.835203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:22.836132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:22.836132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:22.836132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:22.836132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:22.836132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 20:01:22.836132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 20:01:22.836132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 20:01:22.836132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 20:01:22.836132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 20:01:22.836132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 20:01:22.836132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 20:01:22.836132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:22.836132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 604/1208] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 20:01:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:22.841346 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0df998-a692-4ab2-a556-9b1a6fca1f9a None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:22.937037 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0df998-a692-4ab2-a556-9b1a6fca1f9a None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:22.969491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89e9a9-e569-455e-9235-d54b916f5f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:22.970171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89e9a9-e569-455e-9235-d54b916f5f0e None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:22.970453 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89e9a9-e569-455e-9235-d54b916f5f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:22.970725 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89e9a9-e569-455e-9235-d54b916f5f0e None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:22.971524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89e9a9-e569-455e-9235-d54b916f5f0e None admin] RBAC: Adding query filter params (name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:22.971829 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89e9a9-e569-455e-9235-d54b916f5f0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:22.973196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89e9a9-e569-455e-9235-d54b916f5f0e None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:22.981572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 605/1209] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 20:01:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:22.987261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f8bd5-376b-4bc3-94bf-413a88dc87e9 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:23.081662 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f8bd5-376b-4bc3-94bf-413a88dc87e9 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:23.114433 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44db4c7-ec55-4675-a39d-4d0345a0f68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:23.115229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44db4c7-ec55-4675-a39d-4d0345a0f68b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:23.115515 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44db4c7-ec55-4675-a39d-4d0345a0f68b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:23.115791 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44db4c7-ec55-4675-a39d-4d0345a0f68b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:23.126252 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44db4c7-ec55-4675-a39d-4d0345a0f68b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:23.127692 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44db4c7-ec55-4675-a39d-4d0345a0f68b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:23.144506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-a44db4c7-ec55-4675-a39d-4d0345a0f68b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:23.144506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:23.144506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:23.144506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:23.144506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:23.144506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:23.144506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:23.144506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:23.144506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:23.144506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:23.144506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:23.144506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:23.144506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 20:01:23.144506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 20:01:23.144506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 20:01:23.145587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 20:01:23.145587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:23.145587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:23.145587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:23.145587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:23.145587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:23.145587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:23.145587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:23.145587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:01:23.145587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:23.145587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:23.145587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:23.145587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:23.145587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:23.145587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:23.145587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:23.145587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:23.146566 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 20:01:23.146566 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 20:01:23.146566 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:23.146566 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:23.146566 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 605/1210] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:23.151554 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89e9a9-e569-455e-9235-d54b916f5f0e None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:23.247746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89e9a9-e569-455e-9235-d54b916f5f0e None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:23.278913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9835a-ea7e-4c46-849c-3d1cac2a1498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:23.279562 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9835a-ea7e-4c46-849c-3d1cac2a1498 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:23.280329 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9835a-ea7e-4c46-849c-3d1cac2a1498 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:23.280329 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9835a-ea7e-4c46-849c-3d1cac2a1498 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:23.280814 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9835a-ea7e-4c46-849c-3d1cac2a1498 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:23.281122 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9835a-ea7e-4c46-849c-3d1cac2a1498 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:23.282507 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9835a-ea7e-4c46-849c-3d1cac2a1498 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:23.294400 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 606/1211] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 20:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:23.300217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44db4c7-ec55-4675-a39d-4d0345a0f68b None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:23.393573 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44db4c7-ec55-4675-a39d-4d0345a0f68b None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:23.427564 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d1b78-f353-4cc6-a8ed-5c303fd46c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:23.428425 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d1b78-f353-4cc6-a8ed-5c303fd46c92 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:23.428711 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d1b78-f353-4cc6-a8ed-5c303fd46c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:23.428989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d1b78-f353-4cc6-a8ed-5c303fd46c92 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:23.436273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d1b78-f353-4cc6-a8ed-5c303fd46c92 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:23.437702 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d1b78-f353-4cc6-a8ed-5c303fd46c92 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:23.452055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-d05d1b78-f353-4cc6-a8ed-5c303fd46c92 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 10 20:01:23.452055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:23.452055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:23.452055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:23.452055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:23.452055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:23.452055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:23.452055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:23.452055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:23.452055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:23.452055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:23.452055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:23.452055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 20:01:23.452055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 20:01:23.452055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 20:01:23.452055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 20:01:23.453190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:23.453190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:23.453190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 20:01:23.453190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 20:01:23.453190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 20:01:23.453190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 20:01:23.453190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:23.453190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:23.453190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:23.453190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:23.453190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:23.453190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:01:23.453190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:23.453190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:23.453190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:23.453190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:23.453190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:23.453190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:23.454181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:23.454181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:23.454181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 20:01:23.454181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 20:01:23.454181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 20:01:23.454181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 20:01:23.454181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 20:01:23.454181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 20:01:23.454181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 10 20:01:23.454181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:23.454181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 606/1212] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:23.459229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9835a-ea7e-4c46-849c-3d1cac2a1498 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:23.556444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9835a-ea7e-4c46-849c-3d1cac2a1498 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:23.594020 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfee2c9-0a83-4a1b-b184-c4a82c6ee6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:23.594879 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfee2c9-0a83-4a1b-b184-c4a82c6ee6ee None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:23.595151 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfee2c9-0a83-4a1b-b184-c4a82c6ee6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:23.595428 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfee2c9-0a83-4a1b-b184-c4a82c6ee6ee None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:23.596270 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfee2c9-0a83-4a1b-b184-c4a82c6ee6ee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:23.596582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfee2c9-0a83-4a1b-b184-c4a82c6ee6ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:23.598013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfee2c9-0a83-4a1b-b184-c4a82c6ee6ee None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:23.615300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 607/1213] 10.222.0.65 () {56 vars in 1143 bytes} [Tue May 10 20:01:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:23.621100 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d1b78-f353-4cc6-a8ed-5c303fd46c92 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:23.717978 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d1b78-f353-4cc6-a8ed-5c303fd46c92 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:23.750892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b75ef9-f84b-43bf-9f8c-170b2529ffcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:23.751527 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b75ef9-f84b-43bf-9f8c-170b2529ffcb None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:23.751809 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b75ef9-f84b-43bf-9f8c-170b2529ffcb None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:23.752084 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b75ef9-f84b-43bf-9f8c-170b2529ffcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:23.815661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b75ef9-f84b-43bf-9f8c-170b2529ffcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:23.817016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b75ef9-f84b-43bf-9f8c-170b2529ffcb None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:23.915804 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 607/1214] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 20:01:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 20:01:23.921611 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfee2c9-0a83-4a1b-b184-c4a82c6ee6ee None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:24.017714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfee2c9-0a83-4a1b-b184-c4a82c6ee6ee None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:24.049545 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a630f-0e6c-4876-a295-02ebc8f78b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:24.050352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a630f-0e6c-4876-a295-02ebc8f78b23 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:24.050639 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a630f-0e6c-4876-a295-02ebc8f78b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:24.050917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a630f-0e6c-4876-a295-02ebc8f78b23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:24.061196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a630f-0e6c-4876-a295-02ebc8f78b23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:24.062705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a630f-0e6c-4876-a295-02ebc8f78b23 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:24.080217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-e93a630f-0e6c-4876-a295-02ebc8f78b23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:24.080217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:24.080217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:24.080217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:24.080217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:24.080217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:24.080217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:24.080217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:24.080217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:24.080217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:24.080217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:24.080217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:24.080217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 20:01:24.080217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 20:01:24.080217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 20:01:24.081293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 20:01:24.081293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:24.081293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:24.081293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:24.081293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:24.081293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:24.081293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:24.081293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:24.081293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 20:01:24.081293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:24.081293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:24.081293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:24.081293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:24.081293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:24.081293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:24.081293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:24.081293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:24.085270 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 20:01:24.085270 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 20:01:24.085270 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:24.085270 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 20:01:24.085270 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 608/1215] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:24.086981 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b75ef9-f84b-43bf-9f8c-170b2529ffcb None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:24.179697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b75ef9-f84b-43bf-9f8c-170b2529ffcb None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:24.213560 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f01820-3ea1-4524-8fd0-bc5bcf002489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:24.214272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f01820-3ea1-4524-8fd0-bc5bcf002489 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:24.214561 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f01820-3ea1-4524-8fd0-bc5bcf002489 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:24.214841 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f01820-3ea1-4524-8fd0-bc5bcf002489 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:24.215582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f01820-3ea1-4524-8fd0-bc5bcf002489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:24.215887 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f01820-3ea1-4524-8fd0-bc5bcf002489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:24.217250 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f01820-3ea1-4524-8fd0-bc5bcf002489 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:24.228252 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 608/1216] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 20:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:24.234225 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a630f-0e6c-4876-a295-02ebc8f78b23 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:24.330285 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a630f-0e6c-4876-a295-02ebc8f78b23 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:24.363025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad7fbe-078b-413c-a709-608d5385b4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:24.363804 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad7fbe-078b-413c-a709-608d5385b4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:24.364089 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad7fbe-078b-413c-a709-608d5385b4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:24.364368 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad7fbe-078b-413c-a709-608d5385b4f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:24.371510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad7fbe-078b-413c-a709-608d5385b4f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:24.372946 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad7fbe-078b-413c-a709-608d5385b4f7 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:24.386395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-40ad7fbe-078b-413c-a709-608d5385b4f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 20:01:24.386395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:24.386395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:24.386395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:24.386395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:24.386395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:24.386395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:24.386395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:24.386395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:24.386395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:24.386395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:24.386395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:24.386395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 20:01:24.386395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 20:01:24.386395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 20:01:24.387366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 20:01:24.387366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:24.387366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:24.387366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:24.387366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:24.387366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:24.387366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:24.387366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:24.387366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 20:01:24.387366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:24.387366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:24.387366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:24.387366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:24.387366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:24.387366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:24.387366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:24.387366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:24.388671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 20:01:24.388671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 20:01:24.388671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 20:01:24.388671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 20:01:24.388671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 20:01:24.388671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 20:01:24.388671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 20:01:24.388671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 20:01:24.388671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 609/1217] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 20:01:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:24.393503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f01820-3ea1-4524-8fd0-bc5bcf002489 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:24.495923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f01820-3ea1-4524-8fd0-bc5bcf002489 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:24.527608 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c3cc6-5f7d-4e05-bb03-9e0e91091170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:24.528263 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c3cc6-5f7d-4e05-bb03-9e0e91091170 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:24.528549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c3cc6-5f7d-4e05-bb03-9e0e91091170 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:24.528829 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c3cc6-5f7d-4e05-bb03-9e0e91091170 None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:24.529674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c3cc6-5f7d-4e05-bb03-9e0e91091170 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:24.529932 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c3cc6-5f7d-4e05-bb03-9e0e91091170 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:24.531395 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c3cc6-5f7d-4e05-bb03-9e0e91091170 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:24.543706 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 609/1218] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 20:01:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:24.549891 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad7fbe-078b-413c-a709-608d5385b4f7 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:24.646108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad7fbe-078b-413c-a709-608d5385b4f7 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:24.678704 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0096aa-425d-4aa4-9d97-7d88efd639dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:24.679356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0096aa-425d-4aa4-9d97-7d88efd639dc None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:24.679640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0096aa-425d-4aa4-9d97-7d88efd639dc None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:24.679917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0096aa-425d-4aa4-9d97-7d88efd639dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:24.680744 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0096aa-425d-4aa4-9d97-7d88efd639dc None admin] RBAC: Adding query filter params (role.id=55528a93cfa64162b2f9574fb5cdde79, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=022093dea10d46478b3016395c712d93) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:24.681059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0096aa-425d-4aa4-9d97-7d88efd639dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:24.683070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0096aa-425d-4aa4-9d97-7d88efd639dc None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:24.688586 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 610/1219] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 20:01:24 2022] GET /identity/v3/role_assignments?user.id=022093dea10d46478b3016395c712d93&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=55528a93cfa64162b2f9574fb5cdde79 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:26.860968 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514f390-271b-4f17-b95d-5ba3c5c39e1c None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:26.861543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514f390-271b-4f17-b95d-5ba3c5c39e1c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:26.861543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514f390-271b-4f17-b95d-5ba3c5c39e1c None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:26.862896 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 610/1220] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:01:26.868150 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb0709-cb64-41a1-8109-64e063b0bd88 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:26.868553 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb0709-cb64-41a1-8109-64e063b0bd88 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:26.870248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb0709-cb64-41a1-8109-64e063b0bd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:26.885615 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-4dfb0709-cb64-41a1-8109-64e063b0bd88 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:01:26.886826 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-4dfb0709-cb64-41a1-8109-64e063b0bd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:01:26.951104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 611/1221] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:01:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 20:01:26.974030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514f390-271b-4f17-b95d-5ba3c5c39e1c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:27.071214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514f390-271b-4f17-b95d-5ba3c5c39e1c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:27.104165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b3213-79f1-436a-8ac2-77e6fe39cef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:27.104939 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b3213-79f1-436a-8ac2-77e6fe39cef2 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:27.105220 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b3213-79f1-436a-8ac2-77e6fe39cef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:27.105491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b3213-79f1-436a-8ac2-77e6fe39cef2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:27.117336 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b3213-79f1-436a-8ac2-77e6fe39cef2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 20:01:27.123942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-e34b3213-79f1-436a-8ac2-77e6fe39cef2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 20:01:27.123942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:27.123942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:27.123942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:27.123942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:27.123942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:27.123942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:27.123942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:27.123942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:27.123942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:27.123942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:27.123942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:27.123942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 20:01:27.123942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 20:01:27.123942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 20:01:27.123942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 20:01:27.125078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 20:01:27.125078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 20:01:27.125078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 20:01:27.125078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 20:01:27.125078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:27.125078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:27.125078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:27.125078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:27.125078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:27.125078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:27.125078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:27.125078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:01:27.125078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:27.125078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:27.125078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:27.125078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:27.126262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:27.126262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:27.126262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:27.126262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:27.126262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 20:01:27.126262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 20:01:27.126262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 20:01:27.126262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 20:01:27.126262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 20:01:27.126262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 20:01:27.126262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 20:01:27.126262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:27.126262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 611/1222] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 20:01:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:27.131153 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb0709-cb64-41a1-8109-64e063b0bd88 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:27.225636 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb0709-cb64-41a1-8109-64e063b0bd88 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:27.258476 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c14ef-e7b1-4403-9d88-ddf526afd8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:27.259138 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c14ef-e7b1-4403-9d88-ddf526afd8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:27.260047 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c14ef-e7b1-4403-9d88-ddf526afd8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:27.260282 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c14ef-e7b1-4403-9d88-ddf526afd8a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:27.261027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c14ef-e7b1-4403-9d88-ddf526afd8a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:27.261335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c14ef-e7b1-4403-9d88-ddf526afd8a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:27.262741 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c14ef-e7b1-4403-9d88-ddf526afd8a2 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:27.270937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 612/1223] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 20:01:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 20:01:27.276932 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b3213-79f1-436a-8ac2-77e6fe39cef2 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:27.373448 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b3213-79f1-436a-8ac2-77e6fe39cef2 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:27.404863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae5673-6b0b-46c1-8af3-08b540776a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:27.405695 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae5673-6b0b-46c1-8af3-08b540776a19 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:27.405979 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae5673-6b0b-46c1-8af3-08b540776a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:27.406294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae5673-6b0b-46c1-8af3-08b540776a19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:27.417003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae5673-6b0b-46c1-8af3-08b540776a19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:27.418639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae5673-6b0b-46c1-8af3-08b540776a19 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:27.436417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-29ae5673-6b0b-46c1-8af3-08b540776a19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:27.436417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:27.436417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:27.436417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:27.436417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:27.436417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:27.436417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:27.436417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:27.436417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:27.436417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:27.436417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:27.436417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:27.436417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 20:01:27.436417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 20:01:27.436417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 20:01:27.437505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 20:01:27.437505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:27.437505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:27.437505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:27.437505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:27.437505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:27.437505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:27.437505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:27.437505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:01:27.437505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:27.437505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:27.437505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:27.437505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:27.437505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:27.437505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:27.437505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:27.437505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:27.438813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 20:01:27.438813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 20:01:27.438813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:27.438813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:27.438813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 612/1224] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:27.443350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c14ef-e7b1-4403-9d88-ddf526afd8a2 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:27.539329 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c14ef-e7b1-4403-9d88-ddf526afd8a2 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:27.572097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea1946-4744-4449-97da-efe018edabf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:27.572753 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea1946-4744-4449-97da-efe018edabf5 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:27.573039 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea1946-4744-4449-97da-efe018edabf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:27.573316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea1946-4744-4449-97da-efe018edabf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:27.574040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea1946-4744-4449-97da-efe018edabf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:27.574383 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea1946-4744-4449-97da-efe018edabf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:27.575744 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea1946-4744-4449-97da-efe018edabf5 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:27.586984 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 613/1225] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 20:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:27.592862 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae5673-6b0b-46c1-8af3-08b540776a19 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:27.689050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae5673-6b0b-46c1-8af3-08b540776a19 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:27.721584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e71271-307d-4c8a-b646-09f644ed39c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:27.722417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e71271-307d-4c8a-b646-09f644ed39c1 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:27.722699 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e71271-307d-4c8a-b646-09f644ed39c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:27.722971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e71271-307d-4c8a-b646-09f644ed39c1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:27.731074 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e71271-307d-4c8a-b646-09f644ed39c1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:27.732504 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e71271-307d-4c8a-b646-09f644ed39c1 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:27.746274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-e0e71271-307d-4c8a-b646-09f644ed39c1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 10 20:01:27.746274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:27.746274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:27.746274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:27.746274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:27.746274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:27.746274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:27.746274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:27.746274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:27.746274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:27.746274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:27.746274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:27.746274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 20:01:27.746274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 20:01:27.746274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 20:01:27.746274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 20:01:27.747359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:27.747359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:27.747359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 20:01:27.747359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 20:01:27.747359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 20:01:27.747359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 20:01:27.747359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:27.747359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:27.747359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:27.747359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:27.747359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:27.747359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:01:27.747359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:27.747359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:27.747359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:27.747359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:27.747359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:27.747359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:27.748369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:27.748369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:27.748369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 20:01:27.748369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 20:01:27.748369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 20:01:27.748369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 20:01:27.748369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 20:01:27.748369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 20:01:27.748369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 10 20:01:27.748369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:27.748369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 613/1226] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:27.753520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea1946-4744-4449-97da-efe018edabf5 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:27.848622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea1946-4744-4449-97da-efe018edabf5 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:27.881594 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0cfb5-4ecb-4bb8-9e1c-0510a3c80173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:27.882288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0cfb5-4ecb-4bb8-9e1c-0510a3c80173 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:27.882577 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0cfb5-4ecb-4bb8-9e1c-0510a3c80173 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:27.882855 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0cfb5-4ecb-4bb8-9e1c-0510a3c80173 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:27.883729 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0cfb5-4ecb-4bb8-9e1c-0510a3c80173 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:27.884040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0cfb5-4ecb-4bb8-9e1c-0510a3c80173 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:27.885502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0cfb5-4ecb-4bb8-9e1c-0510a3c80173 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:27.902211 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 614/1227] 10.222.0.65 () {56 vars in 1143 bytes} [Tue May 10 20:01:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:27.907962 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e71271-307d-4c8a-b646-09f644ed39c1 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:28.003218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e71271-307d-4c8a-b646-09f644ed39c1 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:28.035416 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9b0ba-3270-4fed-af4f-2111dacf74d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:28.036201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9b0ba-3270-4fed-af4f-2111dacf74d0 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:28.036484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9b0ba-3270-4fed-af4f-2111dacf74d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:28.036758 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9b0ba-3270-4fed-af4f-2111dacf74d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:28.047692 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9b0ba-3270-4fed-af4f-2111dacf74d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:28.051115 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9b0ba-3270-4fed-af4f-2111dacf74d0 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:28.065812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-8dc9b0ba-3270-4fed-af4f-2111dacf74d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:28.065812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:28.065812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:28.065812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:28.065812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:28.065812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:28.065812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:28.065812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:28.065812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:28.065812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:28.065812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:28.065812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:28.065812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 20:01:28.065812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 20:01:28.065812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 20:01:28.066908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 20:01:28.066908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:28.066908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:28.066908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:28.066908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:28.066908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:28.066908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:28.066908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:28.066908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:01:28.066908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:28.066908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:28.066908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:28.066908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:28.066908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:28.066908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:28.066908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:28.066908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:28.068043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 20:01:28.068043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 20:01:28.068043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:28.068043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:28.068043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 614/1228] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:28.072814 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0cfb5-4ecb-4bb8-9e1c-0510a3c80173 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:28.169205 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0cfb5-4ecb-4bb8-9e1c-0510a3c80173 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:28.202245 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789dd55-e636-460b-bd84-3115ea3317af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:28.202899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789dd55-e636-460b-bd84-3115ea3317af None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:28.203181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789dd55-e636-460b-bd84-3115ea3317af None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:28.203454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789dd55-e636-460b-bd84-3115ea3317af None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:28.204179 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789dd55-e636-460b-bd84-3115ea3317af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:28.204596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789dd55-e636-460b-bd84-3115ea3317af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:28.205955 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789dd55-e636-460b-bd84-3115ea3317af None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:28.217349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 615/1229] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 20:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:28.223116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9b0ba-3270-4fed-af4f-2111dacf74d0 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:28.318215 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9b0ba-3270-4fed-af4f-2111dacf74d0 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:28.351916 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370a8f9-dcf0-4fd3-a6c7-4ea3419be2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:28.352708 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370a8f9-dcf0-4fd3-a6c7-4ea3419be2fd None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:28.353019 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370a8f9-dcf0-4fd3-a6c7-4ea3419be2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:28.353553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370a8f9-dcf0-4fd3-a6c7-4ea3419be2fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:28.361091 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370a8f9-dcf0-4fd3-a6c7-4ea3419be2fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:28.364896 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370a8f9-dcf0-4fd3-a6c7-4ea3419be2fd None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:28.376807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-f370a8f9-dcf0-4fd3-a6c7-4ea3419be2fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 20:01:28.376807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:28.376807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:28.376807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:28.376807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:28.376807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:28.376807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:28.376807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:28.376807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:28.376807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:28.376807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:28.376807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:28.376807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 20:01:28.376807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 20:01:28.376807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 20:01:28.377999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 20:01:28.377999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:28.377999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:28.377999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:28.377999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:28.377999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:28.377999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:28.377999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:28.377999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:01:28.377999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:28.377999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:28.377999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:28.377999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:28.377999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:28.377999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:28.377999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:28.377999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:28.379420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 20:01:28.379420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 20:01:28.379420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 20:01:28.379420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 20:01:28.379420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 20:01:28.379420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 20:01:28.379420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 20:01:28.379420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:28.379420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 615/1230] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 20:01:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:28.383918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789dd55-e636-460b-bd84-3115ea3317af None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:28.483531 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789dd55-e636-460b-bd84-3115ea3317af None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:28.517162 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17e875-b896-447e-bdd2-1c34ec1f20d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:28.517817 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17e875-b896-447e-bdd2-1c34ec1f20d4 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:28.518123 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17e875-b896-447e-bdd2-1c34ec1f20d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:28.518409 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17e875-b896-447e-bdd2-1c34ec1f20d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:28.519284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17e875-b896-447e-bdd2-1c34ec1f20d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:28.519684 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17e875-b896-447e-bdd2-1c34ec1f20d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:28.521185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17e875-b896-447e-bdd2-1c34ec1f20d4 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:28.533110 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 616/1231] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 20:01:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:28.539290 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370a8f9-dcf0-4fd3-a6c7-4ea3419be2fd None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:28.636012 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370a8f9-dcf0-4fd3-a6c7-4ea3419be2fd None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:28.675428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96958f7-6a79-441b-8870-75783649fbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:28.676301 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96958f7-6a79-441b-8870-75783649fbc3 None admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:28.676576 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96958f7-6a79-441b-8870-75783649fbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:28.676845 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96958f7-6a79-441b-8870-75783649fbc3 None admin] PATH_INFO: `/v3/projects/a006477d1df94390b599621446bd0f02/users/022093dea10d46478b3016395c712d93/roles/55528a93cfa64162b2f9574fb5cdde79` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:28.690947 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96958f7-6a79-441b-8870-75783649fbc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a006477d1df94390b599621446bd0f02, user_id=022093dea10d46478b3016395c712d93, role_id=55528a93cfa64162b2f9574fb5cdde79)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:28.692397 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96958f7-6a79-441b-8870-75783649fbc3 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:28.773291 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 616/1232] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 20:01:28 2022] PUT /identity/v3/projects/a006477d1df94390b599621446bd0f02/users/022093dea10d46478b3016395c712d93/roles/55528a93cfa64162b2f9574fb5cdde79 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:01:30.989062 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeed3c-ea5f-4c98-8be7-d2a1b821cac3 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:30.989616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeed3c-ea5f-4c98-8be7-d2a1b821cac3 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:30.989616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeed3c-ea5f-4c98-8be7-d2a1b821cac3 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:30.991004 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 617/1233] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:01:30.996871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d2ed6-277c-4426-9b23-ad35bb59aabb None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:30.997283 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d2ed6-277c-4426-9b23-ad35bb59aabb None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:30.997438 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d2ed6-277c-4426-9b23-ad35bb59aabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:31.014457 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-0c0d2ed6-277c-4426-9b23-ad35bb59aabb None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:01:31.015642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-0c0d2ed6-277c-4426-9b23-ad35bb59aabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:01:31.079694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 617/1234] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:01:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 20:01:31.103886 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aeed3c-ea5f-4c98-8be7-d2a1b821cac3 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:31.201072 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aeed3c-ea5f-4c98-8be7-d2a1b821cac3 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:31.233840 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02e171-c3d1-4eb2-ab23-3fff7e14beeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:31.234656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02e171-c3d1-4eb2-ab23-3fff7e14beeb None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:31.235037 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02e171-c3d1-4eb2-ab23-3fff7e14beeb None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:31.235318 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02e171-c3d1-4eb2-ab23-3fff7e14beeb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:31.245856 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b02e171-c3d1-4eb2-ab23-3fff7e14beeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 20:01:31.253520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-9b02e171-c3d1-4eb2-ab23-3fff7e14beeb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 20:01:31.253520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:31.253520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:31.253520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:31.253520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:31.253520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:31.253520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:31.253520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:31.253520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:31.253520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:31.253520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:31.253520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:31.253520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 20:01:31.253520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 20:01:31.253520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 20:01:31.253520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 20:01:31.254664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 20:01:31.254664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 20:01:31.254664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 20:01:31.254664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 20:01:31.254664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:31.254664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:31.254664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:31.254664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:31.254664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:31.254664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:31.254664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:31.254664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 20:01:31.254664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:31.254664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:31.254664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:31.254664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:31.255506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:31.255506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:31.255506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:31.255506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:31.255506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 20:01:31.255506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 20:01:31.255506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 20:01:31.255506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 20:01:31.255506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 20:01:31.255506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 20:01:31.255506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 20:01:31.255506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 20:01:31.255506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 618/1235] 10.222.0.65 () {56 vars in 1102 bytes} [Tue May 10 20:01:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:31.260793 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d2ed6-277c-4426-9b23-ad35bb59aabb None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:31.364228 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d2ed6-277c-4426-9b23-ad35bb59aabb None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:31.404983 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71772d7e-618a-480f-9b7d-9646507598e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:31.405642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71772d7e-618a-480f-9b7d-9646507598e7 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:31.405922 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71772d7e-618a-480f-9b7d-9646507598e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:31.406226 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71772d7e-618a-480f-9b7d-9646507598e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:31.406969 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71772d7e-618a-480f-9b7d-9646507598e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:31.407272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71772d7e-618a-480f-9b7d-9646507598e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:31.408635 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71772d7e-618a-480f-9b7d-9646507598e7 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:31.417345 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 618/1236] 10.222.0.65 () {56 vars in 1103 bytes} [Tue May 10 20:01:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:31.423121 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02e171-c3d1-4eb2-ab23-3fff7e14beeb None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:31.520565 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02e171-c3d1-4eb2-ab23-3fff7e14beeb None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:31.553555 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80719df6-ea97-4a40-81a9-da91348ef22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:31.554366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80719df6-ea97-4a40-81a9-da91348ef22e None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:31.554652 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80719df6-ea97-4a40-81a9-da91348ef22e None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:31.554927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80719df6-ea97-4a40-81a9-da91348ef22e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:31.565105 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80719df6-ea97-4a40-81a9-da91348ef22e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:31.566692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80719df6-ea97-4a40-81a9-da91348ef22e None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:31.583702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-80719df6-ea97-4a40-81a9-da91348ef22e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:31.583702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:31.583702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:31.583702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:31.583702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:31.583702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:31.583702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:31.583702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:31.583702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:31.583702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:31.583702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:31.583702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:31.583702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 20:01:31.583702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 20:01:31.583702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 20:01:31.584846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 20:01:31.584846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:31.584846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:31.584846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:31.584846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:31.584846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:31.584846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:31.584846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:31.584846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 20:01:31.584846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:31.584846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:31.584846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:31.584846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:31.584846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:31.584846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:31.584846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:31.584846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:31.585813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 20:01:31.585813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 20:01:31.585813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:31.585813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 20:01:31.585813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 619/1237] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:31.591421 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71772d7e-618a-480f-9b7d-9646507598e7 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:31.690373 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71772d7e-618a-480f-9b7d-9646507598e7 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:31.724703 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8fa07b-3033-4ac4-97d2-b88ecb728d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:31.725357 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fa07b-3033-4ac4-97d2-b88ecb728d7a None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:31.725637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fa07b-3033-4ac4-97d2-b88ecb728d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:31.725909 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fa07b-3033-4ac4-97d2-b88ecb728d7a None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:31.726692 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8fa07b-3033-4ac4-97d2-b88ecb728d7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:31.727007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8fa07b-3033-4ac4-97d2-b88ecb728d7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:31.729019 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8fa07b-3033-4ac4-97d2-b88ecb728d7a None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:31.740092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 619/1238] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 20:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:31.746379 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80719df6-ea97-4a40-81a9-da91348ef22e None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:31.843563 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80719df6-ea97-4a40-81a9-da91348ef22e None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:31.876594 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b96e6-6636-4cd5-a3c4-9bcc5b6c49ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:31.879483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b96e6-6636-4cd5-a3c4-9bcc5b6c49ff None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:31.879483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b96e6-6636-4cd5-a3c4-9bcc5b6c49ff None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:31.879483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b96e6-6636-4cd5-a3c4-9bcc5b6c49ff None admin] PATH_INFO: `/v3/users/placement` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:31.886106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b96e6-6636-4cd5-a3c4-9bcc5b6c49ff None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:31.887576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b96e6-6636-4cd5-a3c4-9bcc5b6c49ff None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:31.901275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-398b96e6-6636-4cd5-a3c4-9bcc5b6c49ff None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 10 20:01:31.901275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:31.901275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:31.901275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:31.901275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:31.901275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:31.901275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:31.901275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:31.901275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:31.901275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:31.901275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:31.901275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:31.901275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 20:01:31.901275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 20:01:31.901275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 20:01:31.901275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 20:01:31.903444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:31.903444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:31.903444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 20:01:31.903444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 20:01:31.903444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 20:01:31.903444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 20:01:31.903444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:31.903444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:31.903444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:31.903444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:31.903444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:31.903444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 20:01:31.903444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:31.903444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:31.903444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:31.903444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:31.903444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:31.903444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:31.904442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:31.904442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:31.904442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 20:01:31.904442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 20:01:31.904442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 20:01:31.904442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 20:01:31.904442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 20:01:31.904442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 20:01:31.904442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 10 20:01:31.904442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 20:01:31.904442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 620/1239] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:31.908271 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8fa07b-3033-4ac4-97d2-b88ecb728d7a None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:32.020038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8fa07b-3033-4ac4-97d2-b88ecb728d7a None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:32.052559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e52992-a77b-4fab-9867-ea0a5582c4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:32.053216 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e52992-a77b-4fab-9867-ea0a5582c4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:32.053497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e52992-a77b-4fab-9867-ea0a5582c4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:32.053766 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e52992-a77b-4fab-9867-ea0a5582c4a6 None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:32.054631 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e52992-a77b-4fab-9867-ea0a5582c4a6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:32.054937 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e52992-a77b-4fab-9867-ea0a5582c4a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:32.056992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e52992-a77b-4fab-9867-ea0a5582c4a6 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:32.073469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 620/1240] 10.222.0.65 () {56 vars in 1143 bytes} [Tue May 10 20:01:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:32.079417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b96e6-6636-4cd5-a3c4-9bcc5b6c49ff None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:32.174315 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b96e6-6636-4cd5-a3c4-9bcc5b6c49ff None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:32.206435 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393d23f-8ef1-48dd-9d4e-f991118e4a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:32.207068 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393d23f-8ef1-48dd-9d4e-f991118e4a2c None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:32.207351 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393d23f-8ef1-48dd-9d4e-f991118e4a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:32.207625 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393d23f-8ef1-48dd-9d4e-f991118e4a2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:32.271071 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393d23f-8ef1-48dd-9d4e-f991118e4a2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:32.272512 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393d23f-8ef1-48dd-9d4e-f991118e4a2c None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:32.364241 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 621/1241] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 20:01:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 20:01:32.369946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e52992-a77b-4fab-9867-ea0a5582c4a6 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:32.463946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e52992-a77b-4fab-9867-ea0a5582c4a6 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:32.510449 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace77248-ff35-41a5-8076-b2c93a1b7c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:32.511234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace77248-ff35-41a5-8076-b2c93a1b7c45 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:32.511513 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace77248-ff35-41a5-8076-b2c93a1b7c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:32.511786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace77248-ff35-41a5-8076-b2c93a1b7c45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:32.521378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace77248-ff35-41a5-8076-b2c93a1b7c45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:32.522823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace77248-ff35-41a5-8076-b2c93a1b7c45 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:32.539999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-ace77248-ff35-41a5-8076-b2c93a1b7c45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:32.539999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:32.539999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:32.539999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:32.539999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:32.539999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:32.539999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:32.539999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:32.539999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:32.539999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:32.539999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:32.539999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:32.539999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 20:01:32.539999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 20:01:32.539999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 20:01:32.541760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 20:01:32.541760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:32.541760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:32.541760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:32.541760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:32.541760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:32.541760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:32.541760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:32.541760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:01:32.541760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:32.541760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:32.541760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:32.541760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:32.541760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:32.541760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:32.541760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:32.541760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:32.543090 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 20:01:32.543090 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 20:01:32.543090 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:32.543090 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:32.543090 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 621/1242] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:32.547397 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393d23f-8ef1-48dd-9d4e-f991118e4a2c None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:32.646934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393d23f-8ef1-48dd-9d4e-f991118e4a2c None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:32.680680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31968b97-261b-41a9-bee9-12e32b6037e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:32.681861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31968b97-261b-41a9-bee9-12e32b6037e7 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:32.681861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31968b97-261b-41a9-bee9-12e32b6037e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:32.682058 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31968b97-261b-41a9-bee9-12e32b6037e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:32.682640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31968b97-261b-41a9-bee9-12e32b6037e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:32.683782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31968b97-261b-41a9-bee9-12e32b6037e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:32.684498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31968b97-261b-41a9-bee9-12e32b6037e7 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:32.695850 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 622/1243] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 20:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:32.702383 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace77248-ff35-41a5-8076-b2c93a1b7c45 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:32.810547 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace77248-ff35-41a5-8076-b2c93a1b7c45 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:32.844432 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efcc9f-aa7d-4481-9045-93f9564a4bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:32.845239 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efcc9f-aa7d-4481-9045-93f9564a4bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:32.845609 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efcc9f-aa7d-4481-9045-93f9564a4bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:32.845883 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efcc9f-aa7d-4481-9045-93f9564a4bd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:32.852506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efcc9f-aa7d-4481-9045-93f9564a4bd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:32.854347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efcc9f-aa7d-4481-9045-93f9564a4bd0 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:32.868940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-f0efcc9f-aa7d-4481-9045-93f9564a4bd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 20:01:32.868940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:32.868940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:32.868940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:32.868940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:32.868940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:32.868940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:32.868940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:32.868940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:32.868940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:32.868940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:32.868940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:32.868940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 20:01:32.868940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 20:01:32.868940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 20:01:32.870640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 20:01:32.870640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:32.870640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:32.870640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:32.870640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:32.870640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:32.870640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:32.870640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:32.870640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:01:32.870640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:32.870640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:32.870640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:32.870640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:32.870640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:32.870640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:32.870640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:32.870640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:32.872387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 20:01:32.872387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 20:01:32.872387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 20:01:32.872387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 20:01:32.872387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 20:01:32.872387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 20:01:32.872387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 20:01:32.872387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:32.872387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 622/1244] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 20:01:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:32.876086 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31968b97-261b-41a9-bee9-12e32b6037e7 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:32.981702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31968b97-261b-41a9-bee9-12e32b6037e7 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:33.015867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b28d9-f2d9-48e3-aa8e-841c848f6db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:33.016528 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b28d9-f2d9-48e3-aa8e-841c848f6db4 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:33.016842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b28d9-f2d9-48e3-aa8e-841c848f6db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:33.017316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b28d9-f2d9-48e3-aa8e-841c848f6db4 None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:33.018227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b28d9-f2d9-48e3-aa8e-841c848f6db4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:33.018539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b28d9-f2d9-48e3-aa8e-841c848f6db4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:33.019968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b28d9-f2d9-48e3-aa8e-841c848f6db4 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:33.032106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 623/1245] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 20:01:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:33.038333 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efcc9f-aa7d-4481-9045-93f9564a4bd0 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:33.145626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efcc9f-aa7d-4481-9045-93f9564a4bd0 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:33.181470 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf781f3-2701-481e-968b-af25f689ca76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:33.182135 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf781f3-2701-481e-968b-af25f689ca76 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:33.182422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf781f3-2701-481e-968b-af25f689ca76 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:33.182694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf781f3-2701-481e-968b-af25f689ca76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:33.183586 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf781f3-2701-481e-968b-af25f689ca76 None admin] RBAC: Adding query filter params (role.id=55528a93cfa64162b2f9574fb5cdde79, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=022093dea10d46478b3016395c712d93) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:33.183944 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf781f3-2701-481e-968b-af25f689ca76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:33.185890 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf781f3-2701-481e-968b-af25f689ca76 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:33.191824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 623/1246] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 20:01:33 2022] GET /identity/v3/role_assignments?user.id=022093dea10d46478b3016395c712d93&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=55528a93cfa64162b2f9574fb5cdde79 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:35.472489 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595ca7e-fd8f-41e5-9efe-0c79fc95103b None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:35.473059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595ca7e-fd8f-41e5-9efe-0c79fc95103b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:35.473059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595ca7e-fd8f-41e5-9efe-0c79fc95103b None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:35.474442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 624/1247] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:01:35.480173 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af908eff-c36d-49ad-ac79-7d43ae3011b1 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:35.482811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af908eff-c36d-49ad-ac79-7d43ae3011b1 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:35.482811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af908eff-c36d-49ad-ac79-7d43ae3011b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:35.497128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-af908eff-c36d-49ad-ac79-7d43ae3011b1 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:01:35.500057 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-af908eff-c36d-49ad-ac79-7d43ae3011b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:01:35.563102 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 624/1248] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:01:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 20:01:35.586142 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595ca7e-fd8f-41e5-9efe-0c79fc95103b None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:35.685258 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595ca7e-fd8f-41e5-9efe-0c79fc95103b None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:35.720678 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fe9b9-d402-41d6-b904-77c5c8f8b561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:35.721459 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fe9b9-d402-41d6-b904-77c5c8f8b561 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:35.721737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fe9b9-d402-41d6-b904-77c5c8f8b561 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:35.722004 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fe9b9-d402-41d6-b904-77c5c8f8b561 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:35.732367 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fe9b9-d402-41d6-b904-77c5c8f8b561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 20:01:35.739992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-761fe9b9-d402-41d6-b904-77c5c8f8b561 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 20:01:35.739992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:35.739992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:35.739992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:35.739992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:35.739992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:35.739992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:35.739992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:35.739992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:35.739992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:35.739992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:35.739992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:35.739992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 20:01:35.739992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 20:01:35.739992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 20:01:35.739992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 20:01:35.741032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 20:01:35.741032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 20:01:35.741032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 20:01:35.741032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 20:01:35.741032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:35.741032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:35.741032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:35.741032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:35.741032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:35.741032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:35.741032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:35.741032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 20:01:35.741032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:35.741032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:35.741032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:35.741032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:35.741982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:35.741982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:35.741982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:35.741982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:35.741982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 20:01:35.741982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 20:01:35.741982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 20:01:35.741982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 20:01:35.741982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 20:01:35.741982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 20:01:35.741982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 20:01:35.741982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 20:01:35.741982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 625/1249] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 20:01:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:35.752968 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af908eff-c36d-49ad-ac79-7d43ae3011b1 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:35.840713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af908eff-c36d-49ad-ac79-7d43ae3011b1 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:35.873087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8320819-07e4-4d9a-9cee-161e206cd5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:35.873923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8320819-07e4-4d9a-9cee-161e206cd5ff None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:35.874231 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8320819-07e4-4d9a-9cee-161e206cd5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:35.874504 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8320819-07e4-4d9a-9cee-161e206cd5ff None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:35.875247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8320819-07e4-4d9a-9cee-161e206cd5ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:35.875552 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8320819-07e4-4d9a-9cee-161e206cd5ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:35.876900 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8320819-07e4-4d9a-9cee-161e206cd5ff None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:35.885121 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 625/1250] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 20:01:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:35.890754 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fe9b9-d402-41d6-b904-77c5c8f8b561 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:35.986975 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fe9b9-d402-41d6-b904-77c5c8f8b561 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:36.021769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0acbb-387c-412a-984a-f594dddea24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:36.022611 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0acbb-387c-412a-984a-f594dddea24f None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:36.022894 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0acbb-387c-412a-984a-f594dddea24f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:36.023167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0acbb-387c-412a-984a-f594dddea24f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:36.033660 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0acbb-387c-412a-984a-f594dddea24f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:36.035136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0acbb-387c-412a-984a-f594dddea24f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:36.051392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-4fd0acbb-387c-412a-984a-f594dddea24f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:36.051392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:36.051392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:36.051392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:36.051392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:36.051392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:36.051392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:36.051392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:36.051392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:36.051392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:36.051392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:36.051392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:36.051392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 20:01:36.051392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 20:01:36.051392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 20:01:36.056160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 20:01:36.056160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:36.056160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:36.056160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:36.056160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:36.056160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:36.056160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:36.056160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:36.056160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 20:01:36.056160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:36.056160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:36.056160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:36.056160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:36.056160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:36.056160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:36.056160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:36.056160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:36.057251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 20:01:36.057251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 20:01:36.057251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:36.057251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 20:01:36.057251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 626/1251] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:36.058813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8320819-07e4-4d9a-9cee-161e206cd5ff None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:36.155004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8320819-07e4-4d9a-9cee-161e206cd5ff None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:36.187092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24fa76-1df8-4c3a-b2f3-9a0dec0b7906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:36.187747 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24fa76-1df8-4c3a-b2f3-9a0dec0b7906 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:36.188047 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24fa76-1df8-4c3a-b2f3-9a0dec0b7906 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:36.188314 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24fa76-1df8-4c3a-b2f3-9a0dec0b7906 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:36.189039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24fa76-1df8-4c3a-b2f3-9a0dec0b7906 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:36.189346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24fa76-1df8-4c3a-b2f3-9a0dec0b7906 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:36.190745 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24fa76-1df8-4c3a-b2f3-9a0dec0b7906 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:36.203124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 626/1252] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 20:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:36.208937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0acbb-387c-412a-984a-f594dddea24f None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:36.304878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0acbb-387c-412a-984a-f594dddea24f None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:36.337384 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9888d-ff0f-4390-b1a8-57df8964a82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:36.338200 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9888d-ff0f-4390-b1a8-57df8964a82b None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:36.338487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9888d-ff0f-4390-b1a8-57df8964a82b None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:36.338769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9888d-ff0f-4390-b1a8-57df8964a82b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:36.345884 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9888d-ff0f-4390-b1a8-57df8964a82b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:36.347424 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9888d-ff0f-4390-b1a8-57df8964a82b None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:36.361997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-73c9888d-ff0f-4390-b1a8-57df8964a82b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 10 20:01:36.361997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:36.361997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:36.361997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:36.361997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:36.361997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:36.361997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:36.361997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:36.361997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:36.361997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:36.361997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:36.361997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:36.361997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 20:01:36.361997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 20:01:36.361997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 20:01:36.361997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 20:01:36.363160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:36.363160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:36.363160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 20:01:36.363160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 20:01:36.363160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 20:01:36.363160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 20:01:36.363160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:36.363160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:36.363160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:36.363160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:36.363160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:36.363160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 20:01:36.363160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:36.363160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:36.363160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:36.363160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:36.363160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:36.363160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:36.364342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:36.364342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:36.364342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 20:01:36.364342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 20:01:36.364342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 20:01:36.364342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 20:01:36.364342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 20:01:36.364342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 20:01:36.364342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 10 20:01:36.364342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 20:01:36.364342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 627/1253] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:36.369762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24fa76-1df8-4c3a-b2f3-9a0dec0b7906 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:36.466953 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24fa76-1df8-4c3a-b2f3-9a0dec0b7906 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:36.500885 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b625c7-2eee-4448-9f92-ac1c95983921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:36.501544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b625c7-2eee-4448-9f92-ac1c95983921 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:36.501826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b625c7-2eee-4448-9f92-ac1c95983921 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:36.502123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b625c7-2eee-4448-9f92-ac1c95983921 None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:36.503007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b625c7-2eee-4448-9f92-ac1c95983921 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:36.503310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b625c7-2eee-4448-9f92-ac1c95983921 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:36.504722 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b625c7-2eee-4448-9f92-ac1c95983921 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:36.523001 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 627/1254] 10.222.0.65 () {56 vars in 1143 bytes} [Tue May 10 20:01:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:36.528732 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9888d-ff0f-4390-b1a8-57df8964a82b None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:36.624665 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9888d-ff0f-4390-b1a8-57df8964a82b None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:36.655558 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b8f13-d079-4e85-87c9-8c87ca234a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:36.656234 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b8f13-d079-4e85-87c9-8c87ca234a18 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:36.656508 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b8f13-d079-4e85-87c9-8c87ca234a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:36.656776 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b8f13-d079-4e85-87c9-8c87ca234a18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:36.720045 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b8f13-d079-4e85-87c9-8c87ca234a18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:36.721472 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b8f13-d079-4e85-87c9-8c87ca234a18 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:36.814247 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 628/1255] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 20:01:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 20:01:36.820065 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b625c7-2eee-4448-9f92-ac1c95983921 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:36.915501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b625c7-2eee-4448-9f92-ac1c95983921 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:36.947167 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72464e-ebb3-4f0c-b577-1ee5d886a9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:36.947994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72464e-ebb3-4f0c-b577-1ee5d886a9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:36.948273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72464e-ebb3-4f0c-b577-1ee5d886a9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:36.948549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72464e-ebb3-4f0c-b577-1ee5d886a9b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:36.958034 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72464e-ebb3-4f0c-b577-1ee5d886a9b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:36.959535 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72464e-ebb3-4f0c-b577-1ee5d886a9b9 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:36.977117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-1d72464e-ebb3-4f0c-b577-1ee5d886a9b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:36.977117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:36.977117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:36.977117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:36.977117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:36.977117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:36.977117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:36.977117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:36.977117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:36.977117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:36.977117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:36.977117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:36.977117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 20:01:36.977117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 20:01:36.977117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 20:01:36.978683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 20:01:36.978683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:36.978683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:36.978683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:36.978683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:36.978683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:36.978683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:36.978683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:36.978683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:01:36.978683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:36.978683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:36.978683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:36.978683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:36.978683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:36.978683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:36.978683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:36.978683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:36.979596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 20:01:36.979596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 20:01:36.979596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:36.979596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:36.979596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 628/1256] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:36.984635 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b8f13-d079-4e85-87c9-8c87ca234a18 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:37.076966 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b8f13-d079-4e85-87c9-8c87ca234a18 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:37.109407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b9a09-bdbd-4ea7-88e0-3850a71de9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:37.110053 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b9a09-bdbd-4ea7-88e0-3850a71de9ac None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:37.110363 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b9a09-bdbd-4ea7-88e0-3850a71de9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:37.110637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b9a09-bdbd-4ea7-88e0-3850a71de9ac None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:37.111353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b9a09-bdbd-4ea7-88e0-3850a71de9ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:37.111657 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b9a09-bdbd-4ea7-88e0-3850a71de9ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:37.113072 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b9a09-bdbd-4ea7-88e0-3850a71de9ac None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:37.123817 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 629/1257] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 20:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:37.129422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72464e-ebb3-4f0c-b577-1ee5d886a9b9 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:37.225121 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72464e-ebb3-4f0c-b577-1ee5d886a9b9 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:37.257541 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57515bd5-ce08-41dc-95f3-cd0b2581ff6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:37.258349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57515bd5-ce08-41dc-95f3-cd0b2581ff6c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:37.259166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57515bd5-ce08-41dc-95f3-cd0b2581ff6c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:37.259438 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57515bd5-ce08-41dc-95f3-cd0b2581ff6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:37.265853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57515bd5-ce08-41dc-95f3-cd0b2581ff6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:37.267325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57515bd5-ce08-41dc-95f3-cd0b2581ff6c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:37.281670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-57515bd5-ce08-41dc-95f3-cd0b2581ff6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 20:01:37.281670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:37.281670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:37.281670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:37.281670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:37.281670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:37.281670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:37.281670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:37.281670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:37.281670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:37.281670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:37.281670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:37.281670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 20:01:37.281670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 20:01:37.281670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 20:01:37.287030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 20:01:37.287030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:37.287030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:37.287030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:37.287030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:37.287030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:37.287030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:37.287030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:37.287030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:01:37.287030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:37.287030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:37.287030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:37.287030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:37.287030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:37.287030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:37.287030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:37.287030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:37.288282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 20:01:37.288282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 20:01:37.288282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 20:01:37.288282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 20:01:37.288282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 20:01:37.288282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 20:01:37.288282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 20:01:37.288282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:37.288282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 629/1258] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 20:01:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 20:01:37.288924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b9a09-bdbd-4ea7-88e0-3850a71de9ac None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:37.383976 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b9a09-bdbd-4ea7-88e0-3850a71de9ac None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:37.415755 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5a26b-6c0e-4663-bd4b-77eb65a915f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:37.416411 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5a26b-6c0e-4663-bd4b-77eb65a915f7 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:37.416699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5a26b-6c0e-4663-bd4b-77eb65a915f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:37.416979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5a26b-6c0e-4663-bd4b-77eb65a915f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:37.417820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5a26b-6c0e-4663-bd4b-77eb65a915f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:37.418156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5a26b-6c0e-4663-bd4b-77eb65a915f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:37.419597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5a26b-6c0e-4663-bd4b-77eb65a915f7 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:37.431444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 630/1259] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 20:01:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:37.437482 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57515bd5-ce08-41dc-95f3-cd0b2581ff6c None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:37.533136 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57515bd5-ce08-41dc-95f3-cd0b2581ff6c None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:37.565376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b70bd-52e2-450d-8909-4c1228b39b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:37.566016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b70bd-52e2-450d-8909-4c1228b39b5b None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:37.566328 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b70bd-52e2-450d-8909-4c1228b39b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:37.566606 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b70bd-52e2-450d-8909-4c1228b39b5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:37.567436 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b70bd-52e2-450d-8909-4c1228b39b5b None admin] RBAC: Adding query filter params (role.id=5c87e804d0764309ba08ad6e06b23a87, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=022093dea10d46478b3016395c712d93) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:37.567748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b70bd-52e2-450d-8909-4c1228b39b5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:37.569163 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b70bd-52e2-450d-8909-4c1228b39b5b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:37.575142 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 630/1260] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 20:01:37 2022] GET /identity/v3/role_assignments?user.id=022093dea10d46478b3016395c712d93&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:39.791679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ff235-aebc-41be-846f-fa7b2b42e8b5 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:39.792233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ff235-aebc-41be-846f-fa7b2b42e8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:39.792233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ff235-aebc-41be-846f-fa7b2b42e8b5 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:39.793597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 631/1261] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:01:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:01:39.798893 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f352ad1-20c4-4467-957d-d192ebb9cb4e None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:39.799291 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f352ad1-20c4-4467-957d-d192ebb9cb4e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:39.799456 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f352ad1-20c4-4467-957d-d192ebb9cb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:39.815690 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-1f352ad1-20c4-4467-957d-d192ebb9cb4e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:01:39.816852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-1f352ad1-20c4-4467-957d-d192ebb9cb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:01:39.880576 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 631/1262] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:01:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 20:01:39.902821 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ff235-aebc-41be-846f-fa7b2b42e8b5 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:40.001461 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ff235-aebc-41be-846f-fa7b2b42e8b5 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:40.034484 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153ed41-2ca4-4da2-9a7f-4b3b66aabf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:40.035257 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153ed41-2ca4-4da2-9a7f-4b3b66aabf39 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:40.035603 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153ed41-2ca4-4da2-9a7f-4b3b66aabf39 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:40.035848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153ed41-2ca4-4da2-9a7f-4b3b66aabf39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:40.046285 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153ed41-2ca4-4da2-9a7f-4b3b66aabf39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 20:01:40.053888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-4153ed41-2ca4-4da2-9a7f-4b3b66aabf39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 20:01:40.053888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:40.053888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:40.053888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:40.053888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:40.053888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:40.053888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:40.053888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:40.053888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:40.053888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:40.053888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:40.053888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:40.053888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 20:01:40.053888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 20:01:40.053888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 20:01:40.053888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 20:01:40.055101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 20:01:40.055101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 20:01:40.055101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 20:01:40.055101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 20:01:40.055101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:40.055101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:40.055101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:40.055101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:40.055101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:40.055101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:40.055101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:40.055101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 20:01:40.055101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:40.055101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:40.055101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:40.055101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:40.055986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:40.055986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:40.055986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:40.055986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:40.055986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 20:01:40.055986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 20:01:40.055986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 20:01:40.055986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 20:01:40.055986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 20:01:40.055986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 20:01:40.055986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 20:01:40.055986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 20:01:40.055986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 632/1263] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 20:01:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:40.061203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f352ad1-20c4-4467-957d-d192ebb9cb4e None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:40.155405 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f352ad1-20c4-4467-957d-d192ebb9cb4e None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:40.187930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc19473-73d4-468d-8f54-7545ff1d6a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:40.188641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc19473-73d4-468d-8f54-7545ff1d6a09 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:40.188923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc19473-73d4-468d-8f54-7545ff1d6a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:40.189196 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc19473-73d4-468d-8f54-7545ff1d6a09 None admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:40.189950 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc19473-73d4-468d-8f54-7545ff1d6a09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:40.190283 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc19473-73d4-468d-8f54-7545ff1d6a09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:40.191643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc19473-73d4-468d-8f54-7545ff1d6a09 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:40.200516 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 632/1264] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 20:01:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:40.206414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153ed41-2ca4-4da2-9a7f-4b3b66aabf39 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:40.301959 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153ed41-2ca4-4da2-9a7f-4b3b66aabf39 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:40.335663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387694e7-2168-4f51-b828-5e09d9d701b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:40.337080 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387694e7-2168-4f51-b828-5e09d9d701b4 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:40.337365 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387694e7-2168-4f51-b828-5e09d9d701b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:40.337636 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387694e7-2168-4f51-b828-5e09d9d701b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:40.348090 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387694e7-2168-4f51-b828-5e09d9d701b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:40.349498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387694e7-2168-4f51-b828-5e09d9d701b4 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:40.366572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-387694e7-2168-4f51-b828-5e09d9d701b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:40.366572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:40.366572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:40.366572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:40.366572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:40.366572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:40.366572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:40.366572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:40.366572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:40.366572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:40.366572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:40.366572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:40.366572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 20:01:40.366572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 20:01:40.366572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 20:01:40.367637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 20:01:40.367637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:40.367637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:40.367637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:40.367637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:40.367637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:40.367637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:40.367637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:40.367637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 20:01:40.367637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:40.367637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:40.367637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:40.367637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:40.367637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:40.367637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:40.367637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:40.367637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:40.368746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 20:01:40.368746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 20:01:40.368746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:40.368746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 20:01:40.368746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 633/1265] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 20:01:40.373489 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc19473-73d4-468d-8f54-7545ff1d6a09 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:40.472627 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc19473-73d4-468d-8f54-7545ff1d6a09 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:40.505260 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2d853-95af-4527-827d-3b279127a998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:40.505917 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2d853-95af-4527-827d-3b279127a998 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:40.506227 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2d853-95af-4527-827d-3b279127a998 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:40.506499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2d853-95af-4527-827d-3b279127a998 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:40.507220 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2d853-95af-4527-827d-3b279127a998 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:40.507521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2d853-95af-4527-827d-3b279127a998 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:40.509606 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2d853-95af-4527-827d-3b279127a998 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:40.520911 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 633/1266] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 20:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:40.530196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387694e7-2168-4f51-b828-5e09d9d701b4 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:40.627891 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387694e7-2168-4f51-b828-5e09d9d701b4 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:40.660410 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63a2bc-cff4-4a57-97fe-0978b42aba8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:40.661205 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63a2bc-cff4-4a57-97fe-0978b42aba8b None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:40.661489 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63a2bc-cff4-4a57-97fe-0978b42aba8b None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:40.661766 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63a2bc-cff4-4a57-97fe-0978b42aba8b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:40.669949 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63a2bc-cff4-4a57-97fe-0978b42aba8b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:40.671437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63a2bc-cff4-4a57-97fe-0978b42aba8b None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:40.685760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-ed63a2bc-cff4-4a57-97fe-0978b42aba8b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 10 20:01:40.685760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:40.685760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:40.685760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:40.685760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:40.685760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:40.685760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:40.685760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:40.685760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:40.685760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:40.685760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:40.685760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:40.685760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 20:01:40.685760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 20:01:40.685760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 20:01:40.685760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 20:01:40.692202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:40.692202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:40.692202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 20:01:40.692202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 20:01:40.692202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 20:01:40.692202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 20:01:40.692202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:40.692202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:40.692202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:40.692202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:40.692202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:40.692202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 20:01:40.692202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:40.692202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:40.692202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:40.692202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:40.692202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:40.692202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:40.693406 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:40.693406 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:40.693406 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 20:01:40.693406 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 20:01:40.693406 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 20:01:40.693406 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 20:01:40.693406 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 20:01:40.693406 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 20:01:40.693406 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 10 20:01:40.693406 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 20:01:40.693406 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 634/1267] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:40.695398 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2d853-95af-4527-827d-3b279127a998 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:40.790419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2d853-95af-4527-827d-3b279127a998 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:40.824294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae4fd1-6453-495a-bc41-4edd4014f025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:40.824956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae4fd1-6453-495a-bc41-4edd4014f025 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:40.825238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae4fd1-6453-495a-bc41-4edd4014f025 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:40.825517 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae4fd1-6453-495a-bc41-4edd4014f025 None admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:40.826388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae4fd1-6453-495a-bc41-4edd4014f025 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:40.826699 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae4fd1-6453-495a-bc41-4edd4014f025 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:40.828118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae4fd1-6453-495a-bc41-4edd4014f025 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:40.844501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 634/1268] 10.222.0.65 () {56 vars in 1143 bytes} [Tue May 10 20:01:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:40.850200 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63a2bc-cff4-4a57-97fe-0978b42aba8b None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:40.949333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63a2bc-cff4-4a57-97fe-0978b42aba8b None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:40.981934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3f010-e341-412b-8b46-98a25c1b1b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:40.982756 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3f010-e341-412b-8b46-98a25c1b1b5c None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:40.983042 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3f010-e341-412b-8b46-98a25c1b1b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:40.983318 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3f010-e341-412b-8b46-98a25c1b1b5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:40.994202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3f010-e341-412b-8b46-98a25c1b1b5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:40.995688 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3f010-e341-412b-8b46-98a25c1b1b5c None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:41.012093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-73f3f010-e341-412b-8b46-98a25c1b1b5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:41.012093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:41.012093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:41.012093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:41.012093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:41.012093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:41.012093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:41.012093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:41.012093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:41.012093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:41.012093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:41.012093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:41.012093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 20:01:41.012093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 20:01:41.012093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 20:01:41.013134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 20:01:41.013134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:41.013134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:41.013134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:41.013134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:41.013134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:41.013134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:41.013134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:41.013134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 20:01:41.013134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:41.013134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:41.013134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:41.013134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:41.013134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:41.013134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:41.013134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:41.013134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:41.014421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 20:01:41.014421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 20:01:41.014421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:41.014421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 20:01:41.014421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 635/1269] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:41.019083 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae4fd1-6453-495a-bc41-4edd4014f025 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:41.114052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae4fd1-6453-495a-bc41-4edd4014f025 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:41.147014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9606035-6c63-4f85-bcd6-d2cf202efa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:41.147661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9606035-6c63-4f85-bcd6-d2cf202efa5c None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:41.147946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9606035-6c63-4f85-bcd6-d2cf202efa5c None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:41.148223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9606035-6c63-4f85-bcd6-d2cf202efa5c None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:41.148950 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9606035-6c63-4f85-bcd6-d2cf202efa5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:41.149255 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9606035-6c63-4f85-bcd6-d2cf202efa5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:41.150654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9606035-6c63-4f85-bcd6-d2cf202efa5c None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:41.161426 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 635/1270] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 20:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:41.167622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3f010-e341-412b-8b46-98a25c1b1b5c None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:41.261689 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3f010-e341-412b-8b46-98a25c1b1b5c None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:41.293946 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb768a4-785a-4ef0-925d-75c29a237120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:41.294755 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb768a4-785a-4ef0-925d-75c29a237120 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:41.295037 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb768a4-785a-4ef0-925d-75c29a237120 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:41.295312 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb768a4-785a-4ef0-925d-75c29a237120 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:41.301797 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb768a4-785a-4ef0-925d-75c29a237120 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:41.303264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb768a4-785a-4ef0-925d-75c29a237120 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:41.317014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-6cb768a4-785a-4ef0-925d-75c29a237120 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 20:01:41.317014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:41.317014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:41.317014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:41.317014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:41.317014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:41.317014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:41.317014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:41.317014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:41.317014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:41.317014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:41.317014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:41.317014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 20:01:41.317014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 20:01:41.317014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 20:01:41.318184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 20:01:41.318184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:41.318184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:41.318184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:41.318184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:41.318184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:41.318184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:41.318184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:41.318184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 20:01:41.318184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:41.318184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:41.318184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:41.318184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:41.318184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:41.318184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:41.318184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:41.318184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:41.319288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 20:01:41.319288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 20:01:41.319288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 20:01:41.319288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 20:01:41.319288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 20:01:41.319288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 20:01:41.319288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 20:01:41.319288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 20:01:41.319288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 636/1271] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 20:01:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:41.324081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9606035-6c63-4f85-bcd6-d2cf202efa5c None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:41.421511 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9606035-6c63-4f85-bcd6-d2cf202efa5c None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:41.454400 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376ba16-57e9-493b-aade-09a8b78eabe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:41.455063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376ba16-57e9-493b-aade-09a8b78eabe1 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:41.455345 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376ba16-57e9-493b-aade-09a8b78eabe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:41.455616 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376ba16-57e9-493b-aade-09a8b78eabe1 None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:41.456403 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376ba16-57e9-493b-aade-09a8b78eabe1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:41.456710 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376ba16-57e9-493b-aade-09a8b78eabe1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:41.458163 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376ba16-57e9-493b-aade-09a8b78eabe1 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:41.470304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 636/1272] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 20:01:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:41.476136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb768a4-785a-4ef0-925d-75c29a237120 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:41.572354 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb768a4-785a-4ef0-925d-75c29a237120 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:41.605421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6113aa-701c-45b5-afbd-a5c3f49022cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:41.606320 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6113aa-701c-45b5-afbd-a5c3f49022cd None admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:41.606603 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6113aa-701c-45b5-afbd-a5c3f49022cd None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:41.606873 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6113aa-701c-45b5-afbd-a5c3f49022cd None admin] PATH_INFO: `/v3/projects/a006477d1df94390b599621446bd0f02/users/022093dea10d46478b3016395c712d93/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:41.608821 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6113aa-701c-45b5-afbd-a5c3f49022cd None admin] RBAC: Authorizing `identity:create_grant(project_id=a006477d1df94390b599621446bd0f02, user_id=022093dea10d46478b3016395c712d93, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:41.610270 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6113aa-701c-45b5-afbd-a5c3f49022cd None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:41.733016 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 637/1273] 10.222.0.65 () {58 vars in 1439 bytes} [Tue May 10 20:01:41 2022] PUT /identity/v3/projects/a006477d1df94390b599621446bd0f02/users/022093dea10d46478b3016395c712d93/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:01:43.930834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c3dc4-ba65-4e24-adbc-2e540c6660e7 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:43.931366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c3dc4-ba65-4e24-adbc-2e540c6660e7 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:43.931366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c3dc4-ba65-4e24-adbc-2e540c6660e7 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:43.932740 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 637/1274] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:01:43.940087 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe1e26-b21b-4adf-a902-0074abfd0ccc None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:43.940087 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe1e26-b21b-4adf-a902-0074abfd0ccc None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:43.940087 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe1e26-b21b-4adf-a902-0074abfd0ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:43.955052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-1ffe1e26-b21b-4adf-a902-0074abfd0ccc None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:01:43.956230 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1ffe1e26-b21b-4adf-a902-0074abfd0ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:01:44.019956 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 638/1275] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:01:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 20:01:44.042910 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c3dc4-ba65-4e24-adbc-2e540c6660e7 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:44.139147 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c3dc4-ba65-4e24-adbc-2e540c6660e7 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:44.171535 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d43b84-56c6-4202-9540-4b642643568f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:44.172327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d43b84-56c6-4202-9540-4b642643568f None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:44.172593 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d43b84-56c6-4202-9540-4b642643568f None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:44.172870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d43b84-56c6-4202-9540-4b642643568f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:44.183401 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d43b84-56c6-4202-9540-4b642643568f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 20:01:44.190857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-16d43b84-56c6-4202-9540-4b642643568f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 20:01:44.190857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:44.190857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:44.190857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:44.190857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:44.190857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:44.190857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:44.190857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:44.190857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:44.190857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:44.190857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:44.190857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:44.190857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 20:01:44.190857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 20:01:44.190857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 20:01:44.190857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 20:01:44.193661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 20:01:44.193661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 20:01:44.193661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 20:01:44.193661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 20:01:44.193661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:44.193661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:44.193661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:44.193661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:44.193661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:44.193661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:44.193661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:44.193661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:01:44.193661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:44.193661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:44.193661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:44.193661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:44.194811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:44.194811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:44.194811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:44.194811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:44.194811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 20:01:44.194811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 20:01:44.194811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 20:01:44.194811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 20:01:44.194811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 20:01:44.194811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 20:01:44.194811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 20:01:44.194811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:44.194811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 638/1276] 10.222.0.65 () {56 vars in 1096 bytes} [Tue May 10 20:01:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:44.198039 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe1e26-b21b-4adf-a902-0074abfd0ccc None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:44.294176 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe1e26-b21b-4adf-a902-0074abfd0ccc None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:44.326671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8e22e-1c18-4858-9532-e913c9b9c64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:44.327323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8e22e-1c18-4858-9532-e913c9b9c64d None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:44.327603 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8e22e-1c18-4858-9532-e913c9b9c64d None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:44.327874 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8e22e-1c18-4858-9532-e913c9b9c64d None admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:44.328623 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8e22e-1c18-4858-9532-e913c9b9c64d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:44.328926 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8e22e-1c18-4858-9532-e913c9b9c64d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:44.330307 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8e22e-1c18-4858-9532-e913c9b9c64d None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:44.338622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 639/1277] 10.222.0.65 () {56 vars in 1099 bytes} [Tue May 10 20:01:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:44.344306 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d43b84-56c6-4202-9540-4b642643568f None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:44.441772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d43b84-56c6-4202-9540-4b642643568f None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:44.473998 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d1138-1b17-48e2-a89e-cd06cf48a6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:44.474803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d1138-1b17-48e2-a89e-cd06cf48a6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:44.475087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d1138-1b17-48e2-a89e-cd06cf48a6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:44.475363 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d1138-1b17-48e2-a89e-cd06cf48a6d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:44.486485 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d1138-1b17-48e2-a89e-cd06cf48a6d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:44.487945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d1138-1b17-48e2-a89e-cd06cf48a6d3 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:44.504555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-3b1d1138-1b17-48e2-a89e-cd06cf48a6d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:44.504555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:44.504555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:44.504555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:44.504555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:44.504555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:44.504555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:44.504555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:44.504555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:44.504555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:44.504555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:44.504555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:44.504555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 20:01:44.504555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 20:01:44.504555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 20:01:44.505617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 20:01:44.505617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:44.505617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:44.505617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:44.505617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:44.505617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:44.505617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:44.505617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:44.505617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:01:44.505617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:44.505617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:44.505617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:44.505617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:44.505617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:44.505617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:44.505617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:44.505617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:44.506794 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 20:01:44.506794 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 20:01:44.506794 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:44.506794 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:44.506794 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 639/1278] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:44.511755 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8e22e-1c18-4858-9532-e913c9b9c64d None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:44.617385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8e22e-1c18-4858-9532-e913c9b9c64d None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:44.650492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed4c76-b824-47fd-8a72-13ba0ec36dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:44.651210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed4c76-b824-47fd-8a72-13ba0ec36dcd None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:44.651493 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed4c76-b824-47fd-8a72-13ba0ec36dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:44.651768 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed4c76-b824-47fd-8a72-13ba0ec36dcd None admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:44.652492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed4c76-b824-47fd-8a72-13ba0ec36dcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:44.652796 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed4c76-b824-47fd-8a72-13ba0ec36dcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:44.654180 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed4c76-b824-47fd-8a72-13ba0ec36dcd None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:44.665637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 640/1279] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 20:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:44.671336 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d1138-1b17-48e2-a89e-cd06cf48a6d3 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:44.766188 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d1138-1b17-48e2-a89e-cd06cf48a6d3 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:44.797732 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a045460-6f3c-4279-8469-baf9df759fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:44.798542 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a045460-6f3c-4279-8469-baf9df759fdb None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:44.798827 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a045460-6f3c-4279-8469-baf9df759fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:44.799100 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a045460-6f3c-4279-8469-baf9df759fdb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:44.806314 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a045460-6f3c-4279-8469-baf9df759fdb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:44.807755 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a045460-6f3c-4279-8469-baf9df759fdb None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:44.822214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-6a045460-6f3c-4279-8469-baf9df759fdb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 10 20:01:44.822214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:44.822214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:44.822214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:44.822214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:44.822214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:44.822214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:44.822214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:44.822214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:44.822214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:44.822214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:44.822214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:44.822214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 20:01:44.822214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 20:01:44.822214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 20:01:44.822214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 20:01:44.823388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:44.823388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:44.823388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 20:01:44.823388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 20:01:44.823388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 20:01:44.823388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 20:01:44.823388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:44.823388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:44.823388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:44.823388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:44.823388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:44.823388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:01:44.823388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:44.823388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:44.823388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:44.823388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:44.823388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:44.823388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:44.824439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:44.824439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:44.824439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 20:01:44.824439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 20:01:44.824439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 20:01:44.824439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 20:01:44.824439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 20:01:44.824439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 20:01:44.824439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 10 20:01:44.824439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:44.824439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 640/1280] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:44.829242 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed4c76-b824-47fd-8a72-13ba0ec36dcd None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:44.927073 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed4c76-b824-47fd-8a72-13ba0ec36dcd None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:44.959908 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b529cf8-67a9-47e2-b781-077c54095793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:44.961307 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b529cf8-67a9-47e2-b781-077c54095793 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:44.962134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b529cf8-67a9-47e2-b781-077c54095793 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:44.962421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b529cf8-67a9-47e2-b781-077c54095793 None admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:44.963274 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b529cf8-67a9-47e2-b781-077c54095793 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:44.963588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b529cf8-67a9-47e2-b781-077c54095793 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:44.965021 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b529cf8-67a9-47e2-b781-077c54095793 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:44.981677 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 641/1281] 10.222.0.65 () {56 vars in 1143 bytes} [Tue May 10 20:01:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:44.987415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a045460-6f3c-4279-8469-baf9df759fdb None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:45.081946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a045460-6f3c-4279-8469-baf9df759fdb None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:45.113211 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b63202-f639-4c89-9f60-63001cff5bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:45.113838 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b63202-f639-4c89-9f60-63001cff5bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:45.114142 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b63202-f639-4c89-9f60-63001cff5bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:45.114419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b63202-f639-4c89-9f60-63001cff5bf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:45.178904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b63202-f639-4c89-9f60-63001cff5bf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:45.180902 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b63202-f639-4c89-9f60-63001cff5bf6 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:45.274324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 641/1282] 10.222.0.65 () {58 vars in 1282 bytes} [Tue May 10 20:01:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 20:01:45.280044 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b529cf8-67a9-47e2-b781-077c54095793 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:45.379415 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b529cf8-67a9-47e2-b781-077c54095793 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:45.412310 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b0498-2fd3-403c-9ba5-438e8fbdf538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:45.413099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b0498-2fd3-403c-9ba5-438e8fbdf538 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:45.413385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b0498-2fd3-403c-9ba5-438e8fbdf538 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:45.413658 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b0498-2fd3-403c-9ba5-438e8fbdf538 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:45.424313 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b0498-2fd3-403c-9ba5-438e8fbdf538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:45.425807 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b0498-2fd3-403c-9ba5-438e8fbdf538 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:45.444169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-277b0498-2fd3-403c-9ba5-438e8fbdf538 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:45.444169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:45.444169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:45.444169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:45.444169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:45.444169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:45.444169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:45.444169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:45.444169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:45.444169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:45.444169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:45.444169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:45.444169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 20:01:45.444169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 20:01:45.444169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 20:01:45.445239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 20:01:45.445239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:45.445239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:45.445239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:45.445239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:45.445239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:45.445239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:45.445239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:45.445239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 20:01:45.445239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:45.445239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:45.445239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:45.445239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:45.445239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:45.445239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:45.445239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:45.445239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:45.446222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 20:01:45.446222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 20:01:45.446222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 20:01:45.446222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 20:01:45.446222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 642/1283] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:01:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:45.451211 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b63202-f639-4c89-9f60-63001cff5bf6 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:45.544727 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b63202-f639-4c89-9f60-63001cff5bf6 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:45.577187 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33e037-c848-407f-bed5-e785ecb150f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:45.577840 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33e037-c848-407f-bed5-e785ecb150f8 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:45.578155 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33e037-c848-407f-bed5-e785ecb150f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:45.578436 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33e037-c848-407f-bed5-e785ecb150f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:45.579168 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33e037-c848-407f-bed5-e785ecb150f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:45.579479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33e037-c848-407f-bed5-e785ecb150f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:45.580855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33e037-c848-407f-bed5-e785ecb150f8 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:45.592295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 642/1284] 10.222.0.65 () {56 vars in 1109 bytes} [Tue May 10 20:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:45.598033 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b0498-2fd3-403c-9ba5-438e8fbdf538 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:45.693479 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b0498-2fd3-403c-9ba5-438e8fbdf538 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:45.728529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855b113-bab8-4a9d-898f-9a69ef121de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:45.729337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855b113-bab8-4a9d-898f-9a69ef121de3 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:45.729621 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855b113-bab8-4a9d-898f-9a69ef121de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:45.729892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855b113-bab8-4a9d-898f-9a69ef121de3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:45.737352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855b113-bab8-4a9d-898f-9a69ef121de3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:45.738854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855b113-bab8-4a9d-898f-9a69ef121de3 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:45.753679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application [None req-d855b113-bab8-4a9d-898f-9a69ef121de3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 20:01:45.753679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:45.753679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:45.753679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:45.753679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:45.753679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:45.753679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:45.753679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:45.753679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:45.753679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:45.753679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:45.753679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:45.753679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 20:01:45.753679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 20:01:45.753679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 20:01:45.760093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 20:01:45.760093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:45.760093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:45.760093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:45.760093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:45.760093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:45.760093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:45.760093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:45.760093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application with Lock( May 10 20:01:45.760093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:45.760093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:45.760093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:45.760093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:45.760093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:45.760093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:45.760093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:45.760093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:45.761306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 20:01:45.761306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 20:01:45.761306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 20:01:45.761306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 20:01:45.761306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 20:01:45.761306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 20:01:45.761306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 20:01:45.761306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: ERROR keystone.server.flask.application May 10 20:01:45.761306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 643/1285] 10.222.0.65 () {56 vars in 1111 bytes} [Tue May 10 20:01:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:45.761890 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33e037-c848-407f-bed5-e785ecb150f8 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:45.882149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33e037-c848-407f-bed5-e785ecb150f8 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:45.918826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f188caf-810d-47fc-a253-403579417609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:45.919492 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f188caf-810d-47fc-a253-403579417609 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:45.919778 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f188caf-810d-47fc-a253-403579417609 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:45.920158 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f188caf-810d-47fc-a253-403579417609 None admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:45.920966 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f188caf-810d-47fc-a253-403579417609 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:45.921277 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f188caf-810d-47fc-a253-403579417609 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:45.922748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f188caf-810d-47fc-a253-403579417609 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:45.935540 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 643/1286] 10.222.0.65 () {56 vars in 1148 bytes} [Tue May 10 20:01:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 20:01:45.941840 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855b113-bab8-4a9d-898f-9a69ef121de3 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:46.047778 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855b113-bab8-4a9d-898f-9a69ef121de3 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:46.081734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2837db7-9624-4b6b-9a42-3052fdb162f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:46.082517 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2837db7-9624-4b6b-9a42-3052fdb162f3 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:46.082803 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2837db7-9624-4b6b-9a42-3052fdb162f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:46.083082 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2837db7-9624-4b6b-9a42-3052fdb162f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:46.083934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2837db7-9624-4b6b-9a42-3052fdb162f3 None admin] RBAC: Adding query filter params (role.id=5c87e804d0764309ba08ad6e06b23a87, scope.project.id=a006477d1df94390b599621446bd0f02, user.id=022093dea10d46478b3016395c712d93) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:46.084247 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2837db7-9624-4b6b-9a42-3052fdb162f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:46.085718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2837db7-9624-4b6b-9a42-3052fdb162f3 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:46.092913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 644/1287] 10.222.0.65 () {56 vars in 1374 bytes} [Tue May 10 20:01:45 2022] GET /identity/v3/role_assignments?user.id=022093dea10d46478b3016395c712d93&scope.project.id=a006477d1df94390b599621446bd0f02&role.id=5c87e804d0764309ba08ad6e06b23a87 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:48.345702 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0127d-2cc0-4cac-abaf-99e7ab17e384 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:48.346289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0127d-2cc0-4cac-abaf-99e7ab17e384 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:48.346289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0127d-2cc0-4cac-abaf-99e7ab17e384 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:48.347643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 644/1288] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:01:48.353569 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f65bb7-6d9a-473d-9646-6452f73bdb4c None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:48.354012 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f65bb7-6d9a-473d-9646-6452f73bdb4c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:48.354208 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f65bb7-6d9a-473d-9646-6452f73bdb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:48.370590 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-10f65bb7-6d9a-473d-9646-6452f73bdb4c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:01:48.371830 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-10f65bb7-6d9a-473d-9646-6452f73bdb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:01:48.435746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 645/1289] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:01:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 20:01:48.458110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0127d-2cc0-4cac-abaf-99e7ab17e384 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:48.557469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0127d-2cc0-4cac-abaf-99e7ab17e384 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:48.598789 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5e519-0e78-4c82-9908-e903f81b8f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:48.599503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5e519-0e78-4c82-9908-e903f81b8f47 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:48.599784 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5e519-0e78-4c82-9908-e903f81b8f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:48.600058 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5e519-0e78-4c82-9908-e903f81b8f47 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:48.600716 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5e519-0e78-4c82-9908-e903f81b8f47 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:48.602424 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5e519-0e78-4c82-9908-e903f81b8f47 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:48.614040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-37c5e519-0e78-4c82-9908-e903f81b8f47 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 10 20:01:48.614040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:48.614040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 20:01:48.614040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 20:01:48.614040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 20:01:48.614040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 20:01:48.614040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 20:01:48.614040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 20:01:48.614040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 10 20:01:48.614040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:48.614040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 20:01:48.614040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:48.614040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:48.614040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:48.614040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:48.614040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:48.614040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:48.615320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:48.615320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:48.615320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:48.615320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:48.615320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:48.615320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:48.615320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 20:01:48.615320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 20:01:48.615320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 20:01:48.615320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 20:01:48.615320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:48.615320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:48.615320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:48.615320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:48.615320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:48.615320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:48.616602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:48.616602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:01:48.616602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:48.616602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:48.616602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:48.616602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:48.616602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:48.616602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:48.616602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:48.616602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:48.616602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 20:01:48.616602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 20:01:48.616602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 10 20:01:48.616602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:48.616602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 645/1290] 10.222.0.65 () {56 vars in 1117 bytes} [Tue May 10 20:01:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:48.621354 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f65bb7-6d9a-473d-9646-6452f73bdb4c None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:48.716564 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f65bb7-6d9a-473d-9646-6452f73bdb4c None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:48.748948 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0253fb-5c16-4f85-8c6c-0b8daa84932f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:48.749604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0253fb-5c16-4f85-8c6c-0b8daa84932f None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:48.752912 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0253fb-5c16-4f85-8c6c-0b8daa84932f None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:48.752912 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0253fb-5c16-4f85-8c6c-0b8daa84932f None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:48.752912 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0253fb-5c16-4f85-8c6c-0b8daa84932f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:48.752912 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0253fb-5c16-4f85-8c6c-0b8daa84932f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:48.752912 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0253fb-5c16-4f85-8c6c-0b8daa84932f None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:48.757187 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 646/1291] 10.222.0.65 () {56 vars in 1116 bytes} [Tue May 10 20:01:48 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:48.764255 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5e519-0e78-4c82-9908-e903f81b8f47 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:48.861586 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5e519-0e78-4c82-9908-e903f81b8f47 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:48.895417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b99cbde-1014-4d42-927f-f8915faee7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:48.896062 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b99cbde-1014-4d42-927f-f8915faee7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:48.896346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b99cbde-1014-4d42-927f-f8915faee7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:48.896624 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b99cbde-1014-4d42-927f-f8915faee7e6 None admin] PATH_INFO: `/v3/services` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:48.897373 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b99cbde-1014-4d42-927f-f8915faee7e6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:48.897680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b99cbde-1014-4d42-927f-f8915faee7e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:48.899091 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b99cbde-1014-4d42-927f-f8915faee7e6 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:48.903617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 646/1292] 10.222.0.65 () {56 vars in 1116 bytes} [Tue May 10 20:01:48 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:51.100203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a7dbf-c6fd-4704-84de-e7e8a68a5ac7 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:51.100770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a7dbf-c6fd-4704-84de-e7e8a68a5ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:51.100770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a7dbf-c6fd-4704-84de-e7e8a68a5ac7 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:51.103338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 647/1293] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:01:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:01:51.108594 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a584bb-9847-4de0-8446-6f80534c08ed None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:51.108998 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a584bb-9847-4de0-8446-6f80534c08ed None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:51.109146 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a584bb-9847-4de0-8446-6f80534c08ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:51.128360 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-17a584bb-9847-4de0-8446-6f80534c08ed None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:01:51.130495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-17a584bb-9847-4de0-8446-6f80534c08ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:01:51.198189 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 647/1294] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:01:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 20:01:51.220911 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a7dbf-c6fd-4704-84de-e7e8a68a5ac7 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:51.317397 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a7dbf-c6fd-4704-84de-e7e8a68a5ac7 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:51.349216 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e07ba-70ad-4c52-959c-ae6ae67b1c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:51.349858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e07ba-70ad-4c52-959c-ae6ae67b1c57 None admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:51.350163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e07ba-70ad-4c52-959c-ae6ae67b1c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:51.350484 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e07ba-70ad-4c52-959c-ae6ae67b1c57 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:51.351158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e07ba-70ad-4c52-959c-ae6ae67b1c57 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:51.353445 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e07ba-70ad-4c52-959c-ae6ae67b1c57 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:51.445863 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 648/1295] 10.222.0.65 () {60 vars in 1141 bytes} [Tue May 10 20:01:51 2022] POST /identity/v3/services => generated 246 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:01:53.693790 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9e499-bd72-494c-b53f-fbf5296c3c5a None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:53.694377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9e499-bd72-494c-b53f-fbf5296c3c5a None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:53.694377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9e499-bd72-494c-b53f-fbf5296c3c5a None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:53.695737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 648/1296] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:01:53.700918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d2657-6b3f-43d3-af79-255c40265eb3 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:53.701293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d2657-6b3f-43d3-af79-255c40265eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:53.701596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d2657-6b3f-43d3-af79-255c40265eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:53.717931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-990d2657-6b3f-43d3-af79-255c40265eb3 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:01:53.719151 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-990d2657-6b3f-43d3-af79-255c40265eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:01:53.796257 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 649/1297] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:01:53 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 20:01:53.819523 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9e499-bd72-494c-b53f-fbf5296c3c5a None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:53.914679 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9e499-bd72-494c-b53f-fbf5296c3c5a None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:53.946337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05e2e2-ce7f-4151-ac78-20758ececf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:53.947101 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05e2e2-ce7f-4151-ac78-20758ececf43 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:53.947384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05e2e2-ce7f-4151-ac78-20758ececf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:53.947652 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05e2e2-ce7f-4151-ac78-20758ececf43 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:53.948325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05e2e2-ce7f-4151-ac78-20758ececf43 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:53.949697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05e2e2-ce7f-4151-ac78-20758ececf43 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:53.962419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-9c05e2e2-ce7f-4151-ac78-20758ececf43 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 10 20:01:53.962419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:53.962419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 20:01:53.962419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 20:01:53.962419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 20:01:53.962419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 20:01:53.962419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 20:01:53.962419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 20:01:53.962419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 10 20:01:53.962419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:53.962419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 20:01:53.962419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:53.962419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:53.962419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:53.962419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:53.962419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:53.962419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:53.963752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:53.963752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:53.963752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:53.963752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:53.963752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:53.963752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:53.963752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 20:01:53.963752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 20:01:53.963752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 20:01:53.963752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 20:01:53.963752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:53.963752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:53.963752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:53.963752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:53.963752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:53.963752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:53.964735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:53.964735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:01:53.964735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:53.964735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:53.964735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:53.964735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:53.964735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:53.964735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:53.964735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:53.964735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:53.964735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 20:01:53.964735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 20:01:53.964735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 10 20:01:53.964735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:53.964735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 649/1298] 10.222.0.65 () {56 vars in 1117 bytes} [Tue May 10 20:01:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:53.969946 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d2657-6b3f-43d3-af79-255c40265eb3 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:54.064702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d2657-6b3f-43d3-af79-255c40265eb3 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:54.096843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378c935-35ea-44da-8aec-082f36565274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:54.097489 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378c935-35ea-44da-8aec-082f36565274 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:54.097767 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378c935-35ea-44da-8aec-082f36565274 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:54.098038 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378c935-35ea-44da-8aec-082f36565274 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:54.098772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378c935-35ea-44da-8aec-082f36565274 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:54.099074 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378c935-35ea-44da-8aec-082f36565274 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:54.100492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378c935-35ea-44da-8aec-082f36565274 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:54.106491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 650/1299] 10.222.0.65 () {56 vars in 1116 bytes} [Tue May 10 20:01:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:54.112266 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05e2e2-ce7f-4151-ac78-20758ececf43 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:54.206892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05e2e2-ce7f-4151-ac78-20758ececf43 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:54.239328 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb5676-640e-4552-9c78-fe43827287f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:54.239970 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb5676-640e-4552-9c78-fe43827287f0 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:54.240259 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb5676-640e-4552-9c78-fe43827287f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:54.240539 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb5676-640e-4552-9c78-fe43827287f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:54.241331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb5676-640e-4552-9c78-fe43827287f0 None admin] RBAC: Adding query filter params (interface=public, service_id=30cb88e29bc7430dbb250ca8d75e3d34, region_id=RegionOne) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:54.241644 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb5676-640e-4552-9c78-fe43827287f0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:54.243063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb5676-640e-4552-9c78-fe43827287f0 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:54.247878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 650/1300] 10.222.0.65 () {56 vars in 1251 bytes} [Tue May 10 20:01:54 2022] GET /identity/v3/endpoints?service_id=30cb88e29bc7430dbb250ca8d75e3d34&interface=public®ion_id=RegionOne => generated 194 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 20:01:54.253671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378c935-35ea-44da-8aec-082f36565274 None admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:54.349464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378c935-35ea-44da-8aec-082f36565274 None admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:54.381926 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9a4b2-92fb-4d4a-ba39-e9c38c51ec2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:54.383123 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9a4b2-92fb-4d4a-ba39-e9c38c51ec2c None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:54.383395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9a4b2-92fb-4d4a-ba39-e9c38c51ec2c None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:54.388967 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9a4b2-92fb-4d4a-ba39-e9c38c51ec2c None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:54.388967 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9a4b2-92fb-4d4a-ba39-e9c38c51ec2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:54.388967 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9a4b2-92fb-4d4a-ba39-e9c38c51ec2c None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:54.392313 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 651/1301] 10.222.0.65 () {56 vars in 1087 bytes} [Tue May 10 20:01:54 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:01:56.595599 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcede002-6ef1-4b88-a7ee-51ada1b89b6d None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:56.596142 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcede002-6ef1-4b88-a7ee-51ada1b89b6d None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:56.596142 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcede002-6ef1-4b88-a7ee-51ada1b89b6d None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:56.597545 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 651/1302] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:01:56.603646 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ea1dd-35f5-4b4e-bbb1-2ad81cd39d40 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:56.604080 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ea1dd-35f5-4b4e-bbb1-2ad81cd39d40 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:56.604183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ea1dd-35f5-4b4e-bbb1-2ad81cd39d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:56.620803 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-f06ea1dd-35f5-4b4e-bbb1-2ad81cd39d40 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:01:56.622004 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f06ea1dd-35f5-4b4e-bbb1-2ad81cd39d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:01:56.687101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 652/1303] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:01:56 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 20:01:56.709807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcede002-6ef1-4b88-a7ee-51ada1b89b6d None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:56.806673 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcede002-6ef1-4b88-a7ee-51ada1b89b6d None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:56.838828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a75ec-8bec-4b18-b663-da3317e5f353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:56.839611 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a75ec-8bec-4b18-b663-da3317e5f353 None admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:56.839898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a75ec-8bec-4b18-b663-da3317e5f353 None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:56.840176 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a75ec-8bec-4b18-b663-da3317e5f353 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:56.840832 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a75ec-8bec-4b18-b663-da3317e5f353 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:56.842219 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a75ec-8bec-4b18-b663-da3317e5f353 None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:56.853924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-cf7a75ec-8bec-4b18-b663-da3317e5f353 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 10 20:01:56.853924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:56.853924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 20:01:56.853924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 20:01:56.853924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 20:01:56.853924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 20:01:56.853924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 20:01:56.853924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 20:01:56.853924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 10 20:01:56.853924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:56.853924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 20:01:56.853924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:56.853924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:01:56.853924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:01:56.853924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:01:56.853924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:01:56.853924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:01:56.855160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:01:56.855160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:01:56.855160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:01:56.855160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:01:56.855160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:01:56.855160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:01:56.855160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 20:01:56.855160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 20:01:56.855160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 20:01:56.855160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 20:01:56.855160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:01:56.855160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:01:56.855160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:01:56.855160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:01:56.855160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:01:56.855160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:01:56.856055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:01:56.856055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:01:56.856055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:01:56.856055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:01:56.856055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:01:56.856055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:01:56.856055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:01:56.856055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:01:56.856055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:01:56.856055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:01:56.856055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 20:01:56.856055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 20:01:56.856055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 10 20:01:56.856055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:01:56.856055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 652/1304] 10.222.0.65 () {56 vars in 1117 bytes} [Tue May 10 20:01:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 20:01:56.861158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ea1dd-35f5-4b4e-bbb1-2ad81cd39d40 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:56.955566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ea1dd-35f5-4b4e-bbb1-2ad81cd39d40 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:56.987778 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87c661-9d68-420c-94e7-3ff744d012a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:56.988433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87c661-9d68-420c-94e7-3ff744d012a6 None admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:56.988719 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87c661-9d68-420c-94e7-3ff744d012a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:56.988990 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87c661-9d68-420c-94e7-3ff744d012a6 None admin] PATH_INFO: `/v3/services` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:56.989689 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee87c661-9d68-420c-94e7-3ff744d012a6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:01:56.990008 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee87c661-9d68-420c-94e7-3ff744d012a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:56.991445 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee87c661-9d68-420c-94e7-3ff744d012a6 None admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:56.996927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 653/1305] 10.222.0.65 () {56 vars in 1116 bytes} [Tue May 10 20:01:56 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:01:57.002725 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a75ec-8bec-4b18-b663-da3317e5f353 None admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:01:57.096773 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a75ec-8bec-4b18-b663-da3317e5f353 None admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:01:57.128802 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327db38f-8efd-4ad2-b45a-ecca73b8d81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:01:57.129437 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327db38f-8efd-4ad2-b45a-ecca73b8d81b None admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:01:57.129721 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327db38f-8efd-4ad2-b45a-ecca73b8d81b None admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:01:57.129993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327db38f-8efd-4ad2-b45a-ecca73b8d81b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:01:57.130752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327db38f-8efd-4ad2-b45a-ecca73b8d81b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:01:57.132108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327db38f-8efd-4ad2-b45a-ecca73b8d81b None admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:01:57.258398 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 653/1306] 10.222.0.65 () {60 vars in 1144 bytes} [Tue May 10 20:01:57 2022] POST /identity/v3/endpoints => generated 331 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:07:53.774449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a1f92-a48c-4b09-bd1c-66b32466b6d1 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:07:53.775472 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a1f92-a48c-4b09-bd1c-66b32466b6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:07:53.775977 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a1f92-a48c-4b09-bd1c-66b32466b6d1 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:07:53.783669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3a686-da6d-402b-8c68-47a2661dcfd3 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:07:53.784611 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3a686-da6d-402b-8c68-47a2661dcfd3 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:07:53.785100 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3a686-da6d-402b-8c68-47a2661dcfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:07:53.787724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 654/1307] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:07:53 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:07:53.873875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-92a3a686-da6d-402b-8c68-47a2661dcfd3 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:07:53.875468 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-92a3a686-da6d-402b-8c68-47a2661dcfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:07:54.067703 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 654/1308] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:07:54.099422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7548fe9-20b2-4bcb-b5cf-9d1d46c122b2 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:07:54.099654 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7548fe9-20b2-4bcb-b5cf-9d1d46c122b2 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:07:54.099939 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7548fe9-20b2-4bcb-b5cf-9d1d46c122b2 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:07:54.101625 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 655/1309] 10.222.0.65 () {54 vars in 964 bytes} [Tue May 10 20:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:07:54.107812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e82d19-42ad-4ab0-ba05-705602f30b43 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:07:54.108152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e82d19-42ad-4ab0-ba05-705602f30b43 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:07:54.108440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e82d19-42ad-4ab0-ba05-705602f30b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:07:54.194607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-91e82d19-42ad-4ab0-ba05-705602f30b43 None None] MFA Rules not processed for user `a6aabd1571d04bcfae842626fef5ed6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:07:54.195861 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-91e82d19-42ad-4ab0-ba05-705602f30b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:07:54.284579 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 655/1310] 10.222.0.65 () {58 vars in 1061 bytes} [Tue May 10 20:07:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:07:54.289078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7548fe9-20b2-4bcb-b5cf-9d1d46c122b2 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:07:54.290851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a7548fe9-20b2-4bcb-b5cf-9d1d46c122b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:07:54.407404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7548fe9-20b2-4bcb-b5cf-9d1d46c122b2 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:07:54.486189 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f6710-a428-4bc8-9ae9-0d9aee15cd81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:07:54.487203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f6710-a428-4bc8-9ae9-0d9aee15cd81 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:07:54.487714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f6710-a428-4bc8-9ae9-0d9aee15cd81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:07:54.488202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f6710-a428-4bc8-9ae9-0d9aee15cd81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:07:54.490114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-588f6710-a428-4bc8-9ae9-0d9aee15cd81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:07:54.603753 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f6710-a428-4bc8-9ae9-0d9aee15cd81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:07:54.605282 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f6710-a428-4bc8-9ae9-0d9aee15cd81 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:07:54.625303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 656/1311] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:07:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:07:57.193423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a30ff8-f2cf-4227-adde-34a41bbc0733 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:07:57.193963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a30ff8-f2cf-4227-adde-34a41bbc0733 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:07:57.194144 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a30ff8-f2cf-4227-adde-34a41bbc0733 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:07:57.195404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 656/1312] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:07:57.200785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba978ee-41df-446d-8ebd-3177bac94e84 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:07:57.201379 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba978ee-41df-446d-8ebd-3177bac94e84 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:07:57.201379 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba978ee-41df-446d-8ebd-3177bac94e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:07:57.277034 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-fba978ee-41df-446d-8ebd-3177bac94e84 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:07:57.278329 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-fba978ee-41df-446d-8ebd-3177bac94e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:07:57.391228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 657/1313] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:07:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:07:57.444991 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3281c7f-98f4-4ed8-be0d-282250cb63d3 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:07:57.445354 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3281c7f-98f4-4ed8-be0d-282250cb63d3 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:07:57.445699 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3281c7f-98f4-4ed8-be0d-282250cb63d3 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:07:57.447207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 657/1314] 10.222.0.65 () {54 vars in 964 bytes} [Tue May 10 20:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:07:57.453285 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f3ef8-ae11-4975-94aa-17cdbfef3998 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:07:57.453644 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f3ef8-ae11-4975-94aa-17cdbfef3998 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:07:57.453994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f3ef8-ae11-4975-94aa-17cdbfef3998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:07:57.529418 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-6b2f3ef8-ae11-4975-94aa-17cdbfef3998 None None] MFA Rules not processed for user `a6aabd1571d04bcfae842626fef5ed6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:07:57.530310 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-6b2f3ef8-ae11-4975-94aa-17cdbfef3998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:07:57.611352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 658/1315] 10.222.0.65 () {58 vars in 1061 bytes} [Tue May 10 20:07:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:07:57.621706 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3281c7f-98f4-4ed8-be0d-282250cb63d3 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:07:57.623321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a3281c7f-98f4-4ed8-be0d-282250cb63d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:07:57.728169 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3281c7f-98f4-4ed8-be0d-282250cb63d3 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:07:57.816755 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6835c0-c2ee-4519-9b1a-cb7116d26487 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:07:57.817414 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6835c0-c2ee-4519-9b1a-cb7116d26487 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:07:57.817758 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6835c0-c2ee-4519-9b1a-cb7116d26487 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:07:57.818114 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6835c0-c2ee-4519-9b1a-cb7116d26487 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:07:57.820107 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4b6835c0-c2ee-4519-9b1a-cb7116d26487 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:07:57.944679 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6835c0-c2ee-4519-9b1a-cb7116d26487 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:07:57.946473 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6835c0-c2ee-4519-9b1a-cb7116d26487 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:07:57.969261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 658/1316] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:07:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:08:01.263678 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678784a4-855d-4455-b8ce-3b3ce57db629 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:01.264263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678784a4-855d-4455-b8ce-3b3ce57db629 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:01.264513 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678784a4-855d-4455-b8ce-3b3ce57db629 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:01.265892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 659/1317] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:01.271546 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a689ec-ffe8-4c55-94be-c48963d45806 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:01.272041 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a689ec-ffe8-4c55-94be-c48963d45806 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:01.272230 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a689ec-ffe8-4c55-94be-c48963d45806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:01.339225 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-76a689ec-ffe8-4c55-94be-c48963d45806 None None] MFA Rules not processed for user `13e9677bc30f49ee88a08bd637b062b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:01.340440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-76a689ec-ffe8-4c55-94be-c48963d45806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:01.437762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 659/1318] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:08:01 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:01.477317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678784a4-855d-4455-b8ce-3b3ce57db629 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:08:01.478954 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-678784a4-855d-4455-b8ce-3b3ce57db629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:01.581227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678784a4-855d-4455-b8ce-3b3ce57db629 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:08:01.676328 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe8147-552d-4396-8fc0-6aa8925ce77d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:08:01.677206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe8147-552d-4396-8fc0-6aa8925ce77d service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:01.677487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe8147-552d-4396-8fc0-6aa8925ce77d service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:01.677778 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe8147-552d-4396-8fc0-6aa8925ce77d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:01.679475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-95fe8147-552d-4396-8fc0-6aa8925ce77d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:01.788504 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe8147-552d-4396-8fc0-6aa8925ce77d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:08:01.789956 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe8147-552d-4396-8fc0-6aa8925ce77d service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:08:01.807612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 660/1319] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:08:01 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:08:04.897329 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0cb01-3b97-4d0e-a6cb-068c9c0ad622 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:04.897896 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0cb01-3b97-4d0e-a6cb-068c9c0ad622 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:04.897896 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0cb01-3b97-4d0e-a6cb-068c9c0ad622 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:04.899303 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 660/1320] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:04.904502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd7829-0cc0-4b97-ab27-df92b46877ed None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:04.904930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd7829-0cc0-4b97-ab27-df92b46877ed None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:04.905068 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd7829-0cc0-4b97-ab27-df92b46877ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:04.968249 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-2efd7829-0cc0-4b97-ab27-df92b46877ed None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:04.969443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-2efd7829-0cc0-4b97-ab27-df92b46877ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:05.063029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 661/1321] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:08:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:05.102669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0cb01-3b97-4d0e-a6cb-068c9c0ad622 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:08:05.105063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-54e0cb01-3b97-4d0e-a6cb-068c9c0ad622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:05.205730 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0cb01-3b97-4d0e-a6cb-068c9c0ad622 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:08:05.278584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e585ad-b06f-4067-9310-4f18a1968ace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:08:05.279236 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e585ad-b06f-4067-9310-4f18a1968ace service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:05.279525 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e585ad-b06f-4067-9310-4f18a1968ace service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:05.279802 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e585ad-b06f-4067-9310-4f18a1968ace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:05.282068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-37e585ad-b06f-4067-9310-4f18a1968ace service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:05.390336 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e585ad-b06f-4067-9310-4f18a1968ace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:08:05.391927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e585ad-b06f-4067-9310-4f18a1968ace service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:08:05.410149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 661/1322] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:08:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:08:08.910996 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dedeb1-81b5-4cc7-81dc-05311f72e44d None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:08.911518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dedeb1-81b5-4cc7-81dc-05311f72e44d None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:08.911518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dedeb1-81b5-4cc7-81dc-05311f72e44d None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:08.912822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 662/1323] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:08.917992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021c20f-9f41-4188-ac8c-edac48e5f57a None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:08.918468 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021c20f-9f41-4188-ac8c-edac48e5f57a None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:08.918593 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021c20f-9f41-4188-ac8c-edac48e5f57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:08.981555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-a021c20f-9f41-4188-ac8c-edac48e5f57a None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:08.982787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a021c20f-9f41-4188-ac8c-edac48e5f57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:09.082894 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 662/1324] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:08:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:09.118278 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dedeb1-81b5-4cc7-81dc-05311f72e44d None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:08:09.119818 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-13dedeb1-81b5-4cc7-81dc-05311f72e44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:09.223431 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dedeb1-81b5-4cc7-81dc-05311f72e44d None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:08:09.296981 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c37c8-fe8e-49a2-9b97-906615783655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:08:09.297631 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c37c8-fe8e-49a2-9b97-906615783655 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:09.297913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c37c8-fe8e-49a2-9b97-906615783655 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:09.298219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c37c8-fe8e-49a2-9b97-906615783655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:09.299895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-100c37c8-fe8e-49a2-9b97-906615783655 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:09.410753 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c37c8-fe8e-49a2-9b97-906615783655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:08:09.412218 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c37c8-fe8e-49a2-9b97-906615783655 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:08:09.433007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 663/1325] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:08:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:08:11.770021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d54ee8-5565-4640-a265-0c746f8cb969 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:11.770543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d54ee8-5565-4640-a265-0c746f8cb969 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:11.770632 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d54ee8-5565-4640-a265-0c746f8cb969 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:11.771984 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 663/1326] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:11.777230 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f475c9-0f16-495e-93b6-28752caf9ac1 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:11.777650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f475c9-0f16-495e-93b6-28752caf9ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:11.777891 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f475c9-0f16-495e-93b6-28752caf9ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:11.844060 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-e1f475c9-0f16-495e-93b6-28752caf9ac1 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:11.845281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-e1f475c9-0f16-495e-93b6-28752caf9ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:11.950239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 664/1327] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:11.985956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d54ee8-5565-4640-a265-0c746f8cb969 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:08:11.987536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-60d54ee8-5565-4640-a265-0c746f8cb969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:12.089690 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d54ee8-5565-4640-a265-0c746f8cb969 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:08:12.161122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367158fc-8660-46c2-8015-32a7829ad83a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:08:12.161768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367158fc-8660-46c2-8015-32a7829ad83a service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:12.162047 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367158fc-8660-46c2-8015-32a7829ad83a service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:12.162346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367158fc-8660-46c2-8015-32a7829ad83a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:12.163993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-367158fc-8660-46c2-8015-32a7829ad83a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:12.270659 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367158fc-8660-46c2-8015-32a7829ad83a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:08:12.272102 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367158fc-8660-46c2-8015-32a7829ad83a service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:08:12.290765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 664/1328] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:08:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:08:14.574444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b091a2a-5dff-44de-9233-2e558961f78c None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:14.574940 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b091a2a-5dff-44de-9233-2e558961f78c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:14.574940 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b091a2a-5dff-44de-9233-2e558961f78c None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:14.576274 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 665/1329] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:14.582416 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7bd9c-7947-4c09-84b8-1253e7ad10a8 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:14.582843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7bd9c-7947-4c09-84b8-1253e7ad10a8 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:14.582988 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7bd9c-7947-4c09-84b8-1253e7ad10a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:14.645202 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-03f7bd9c-7947-4c09-84b8-1253e7ad10a8 None None] MFA Rules not processed for user `13e9677bc30f49ee88a08bd637b062b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:14.646424 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-03f7bd9c-7947-4c09-84b8-1253e7ad10a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:14.741944 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 665/1330] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:08:14 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:14.778894 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b091a2a-5dff-44de-9233-2e558961f78c None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:08:14.781128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8b091a2a-5dff-44de-9233-2e558961f78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:14.882758 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b091a2a-5dff-44de-9233-2e558961f78c None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:08:14.952828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b41a51-f281-46b0-83e8-9f56ec4185bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:08:14.953477 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b41a51-f281-46b0-83e8-9f56ec4185bb service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:14.953759 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b41a51-f281-46b0-83e8-9f56ec4185bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:14.954338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b41a51-f281-46b0-83e8-9f56ec4185bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:14.955993 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-21b41a51-f281-46b0-83e8-9f56ec4185bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:15.064776 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b41a51-f281-46b0-83e8-9f56ec4185bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:08:15.066165 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b41a51-f281-46b0-83e8-9f56ec4185bb service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:08:15.084546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 666/1331] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:08:14 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:08:17.932274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e138697-7440-4eee-8c88-4ef98236cb7a None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:17.932717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e138697-7440-4eee-8c88-4ef98236cb7a None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:17.932812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e138697-7440-4eee-8c88-4ef98236cb7a None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:17.934296 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 666/1332] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:17.939732 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af9655-0417-4b30-9cbc-d5a0e5011c3f None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:17.940135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af9655-0417-4b30-9cbc-d5a0e5011c3f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:17.940290 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af9655-0417-4b30-9cbc-d5a0e5011c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:18.024124 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-48af9655-0417-4b30-9cbc-d5a0e5011c3f None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:18.024124 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-48af9655-0417-4b30-9cbc-d5a0e5011c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:18.121757 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 667/1333] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:08:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:18.159937 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e138697-7440-4eee-8c88-4ef98236cb7a None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:08:18.161493 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-2e138697-7440-4eee-8c88-4ef98236cb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:18.265585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e138697-7440-4eee-8c88-4ef98236cb7a None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:08:18.339519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c174f6e-19da-4af2-9917-8891be1c0af8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:08:18.340180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c174f6e-19da-4af2-9917-8891be1c0af8 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:18.340462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c174f6e-19da-4af2-9917-8891be1c0af8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:18.340734 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c174f6e-19da-4af2-9917-8891be1c0af8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:18.342561 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-1c174f6e-19da-4af2-9917-8891be1c0af8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:18.476536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c174f6e-19da-4af2-9917-8891be1c0af8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:08:18.477998 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c174f6e-19da-4af2-9917-8891be1c0af8 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:08:18.497037 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 667/1334] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:08:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:08:20.793247 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606125a8-74fd-4198-8388-2d958a4ebd04 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:20.793800 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606125a8-74fd-4198-8388-2d958a4ebd04 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:20.793800 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606125a8-74fd-4198-8388-2d958a4ebd04 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:20.795877 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 668/1335] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:08:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:20.803646 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ff05c-f0bd-4aef-b5be-a799fc983e61 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:20.804120 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ff05c-f0bd-4aef-b5be-a799fc983e61 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:20.804467 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ff05c-f0bd-4aef-b5be-a799fc983e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:20.880684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-b18ff05c-f0bd-4aef-b5be-a799fc983e61 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:20.881889 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b18ff05c-f0bd-4aef-b5be-a799fc983e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:20.975300 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 668/1336] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:21.015397 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606125a8-74fd-4198-8388-2d958a4ebd04 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:08:21.016928 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-606125a8-74fd-4198-8388-2d958a4ebd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:21.117382 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606125a8-74fd-4198-8388-2d958a4ebd04 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:08:21.188534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58528a-7931-4e42-9d0a-805b4d588c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:08:21.189185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58528a-7931-4e42-9d0a-805b4d588c15 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:21.189468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58528a-7931-4e42-9d0a-805b4d588c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:21.189744 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58528a-7931-4e42-9d0a-805b4d588c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:21.191422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-3a58528a-7931-4e42-9d0a-805b4d588c15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:21.297841 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58528a-7931-4e42-9d0a-805b4d588c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:08:21.299238 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58528a-7931-4e42-9d0a-805b4d588c15 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:08:21.317226 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 669/1337] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:08:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:08:24.205786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139875a6-8bb1-4bff-8eb8-917f1074e19d None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:24.206330 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139875a6-8bb1-4bff-8eb8-917f1074e19d None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:24.206443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139875a6-8bb1-4bff-8eb8-917f1074e19d None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:24.207738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 669/1338] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:24.212927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7c930-3289-43c2-b08e-045cf51e04b1 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:24.213414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7c930-3289-43c2-b08e-045cf51e04b1 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:24.213590 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7c930-3289-43c2-b08e-045cf51e04b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:24.276977 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-8cb7c930-3289-43c2-b08e-045cf51e04b1 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:24.278257 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8cb7c930-3289-43c2-b08e-045cf51e04b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:24.381667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 670/1339] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:08:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:24.417111 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139875a6-8bb1-4bff-8eb8-917f1074e19d None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:08:24.419312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-139875a6-8bb1-4bff-8eb8-917f1074e19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:24.524929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139875a6-8bb1-4bff-8eb8-917f1074e19d None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:08:24.528509 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07151478-f73a-47bf-9e74-225b522a8b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:08:24.529660 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07151478-f73a-47bf-9e74-225b522a8b3c service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:24.529944 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07151478-f73a-47bf-9e74-225b522a8b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:24.530259 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07151478-f73a-47bf-9e74-225b522a8b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:24.618636 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07151478-f73a-47bf-9e74-225b522a8b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:08:24.620639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07151478-f73a-47bf-9e74-225b522a8b3c service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:08:24.640299 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 670/1340] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:08:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:08:29.077815 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519caf00-7a48-405e-9fe7-f64878704e95 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:29.078696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519caf00-7a48-405e-9fe7-f64878704e95 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:29.078696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519caf00-7a48-405e-9fe7-f64878704e95 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:29.080051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 671/1341] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:29.085227 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6238b2-75ec-4aec-b598-196df02b2a8b None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:29.085682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6238b2-75ec-4aec-b598-196df02b2a8b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:29.085785 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6238b2-75ec-4aec-b598-196df02b2a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:29.144533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-7a6238b2-75ec-4aec-b598-196df02b2a8b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:29.145743 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7a6238b2-75ec-4aec-b598-196df02b2a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:29.219812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 671/1342] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:29.257969 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519caf00-7a48-405e-9fe7-f64878704e95 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:08:29.259745 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-519caf00-7a48-405e-9fe7-f64878704e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:29.273371 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519caf00-7a48-405e-9fe7-f64878704e95 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:08:29.276881 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50855b9-432b-4854-a964-fb9713ee3670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:08:29.277519 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50855b9-432b-4854-a964-fb9713ee3670 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:29.277865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50855b9-432b-4854-a964-fb9713ee3670 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:29.278174 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50855b9-432b-4854-a964-fb9713ee3670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:29.286931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50855b9-432b-4854-a964-fb9713ee3670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:08:29.288345 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50855b9-432b-4854-a964-fb9713ee3670 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:08:29.298110 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 672/1343] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:08:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:08:32.649727 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f0f36-fffd-4bcd-9d79-736d0d580501 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:32.650315 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f0f36-fffd-4bcd-9d79-736d0d580501 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:32.650315 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f0f36-fffd-4bcd-9d79-736d0d580501 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:32.651828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 672/1344] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:32.657646 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42875b4-8615-4c6d-9a37-605700d8d1cc None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:32.658042 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42875b4-8615-4c6d-9a37-605700d8d1cc None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:32.658258 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42875b4-8615-4c6d-9a37-605700d8d1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:32.685706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-a42875b4-8615-4c6d-9a37-605700d8d1cc None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:32.686921 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a42875b4-8615-4c6d-9a37-605700d8d1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:32.695284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 673/1345] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:08:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:32.734620 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f0f36-fffd-4bcd-9d79-736d0d580501 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:08:32.745058 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f0f36-fffd-4bcd-9d79-736d0d580501 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:08:32.749073 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5014162-f6df-4ebe-a850-c108f6e1155a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:08:32.749697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5014162-f6df-4ebe-a850-c108f6e1155a service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:32.749976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5014162-f6df-4ebe-a850-c108f6e1155a service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:32.750275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5014162-f6df-4ebe-a850-c108f6e1155a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:32.759035 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5014162-f6df-4ebe-a850-c108f6e1155a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:08:32.760441 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5014162-f6df-4ebe-a850-c108f6e1155a service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:08:32.770693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 673/1346] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:08:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:08:37.484727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b00675-7b8b-47f7-8dd1-b18b85df5c2c None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:37.485305 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b00675-7b8b-47f7-8dd1-b18b85df5c2c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:37.485305 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b00675-7b8b-47f7-8dd1-b18b85df5c2c None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:37.486680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 674/1347] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:37.492191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02148682-4a4b-473c-b613-f1dabc12a151 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:37.492638 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02148682-4a4b-473c-b613-f1dabc12a151 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:37.492760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02148682-4a4b-473c-b613-f1dabc12a151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:37.510925 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-02148682-4a4b-473c-b613-f1dabc12a151 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:37.512148 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-02148682-4a4b-473c-b613-f1dabc12a151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:37.520627 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 674/1348] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:37.558326 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b00675-7b8b-47f7-8dd1-b18b85df5c2c None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:08:37.569505 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b00675-7b8b-47f7-8dd1-b18b85df5c2c None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:08:37.572981 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37b992-adba-4d5d-97d9-3c8436a3608a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:08:37.573616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37b992-adba-4d5d-97d9-3c8436a3608a service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:37.573901 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37b992-adba-4d5d-97d9-3c8436a3608a service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:37.574218 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37b992-adba-4d5d-97d9-3c8436a3608a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:37.583359 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37b992-adba-4d5d-97d9-3c8436a3608a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:08:37.584786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37b992-adba-4d5d-97d9-3c8436a3608a service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:08:37.594251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 675/1349] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:08:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:08:42.964757 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4bc01-451c-41d0-9685-daf505a6eb0a None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:42.965815 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4bc01-451c-41d0-9685-daf505a6eb0a None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:42.965815 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4bc01-451c-41d0-9685-daf505a6eb0a None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:42.967228 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 675/1350] 10.222.0.65 () {54 vars in 921 bytes} [Tue May 10 20:08:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:42.976078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847b15d-b04c-4a9f-b818-aa9463eca774 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:42.976752 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847b15d-b04c-4a9f-b818-aa9463eca774 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:42.977041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847b15d-b04c-4a9f-b818-aa9463eca774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:43.122843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-0847b15d-b04c-4a9f-b818-aa9463eca774 None None] MFA Rules not processed for user `022093dea10d46478b3016395c712d93`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:43.124174 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0847b15d-b04c-4a9f-b818-aa9463eca774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:43.241175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 676/1351] 10.222.0.65 () {58 vars in 1018 bytes} [Tue May 10 20:08:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:47.711885 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7bf2b-2aed-48fd-a391-124d0bd60798 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:47.712418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7bf2b-2aed-48fd-a391-124d0bd60798 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:47.712418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7bf2b-2aed-48fd-a391-124d0bd60798 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:47.713786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 676/1352] 10.222.0.65 () {54 vars in 921 bytes} [Tue May 10 20:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:47.720153 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695afcdd-4bcd-4cc3-88aa-00f551900f11 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:47.720619 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695afcdd-4bcd-4cc3-88aa-00f551900f11 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:47.720727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695afcdd-4bcd-4cc3-88aa-00f551900f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:47.739324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-695afcdd-4bcd-4cc3-88aa-00f551900f11 None None] MFA Rules not processed for user `022093dea10d46478b3016395c712d93`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:47.740562 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-695afcdd-4bcd-4cc3-88aa-00f551900f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:47.749790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 677/1353] 10.222.0.65 () {58 vars in 1018 bytes} [Tue May 10 20:08:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:49.262834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41d879-94e9-4422-b9d0-3a4b5113f32c None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:49.263358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41d879-94e9-4422-b9d0-3a4b5113f32c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:49.263358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41d879-94e9-4422-b9d0-3a4b5113f32c None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:49.264783 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 677/1354] 10.222.0.65 () {54 vars in 921 bytes} [Tue May 10 20:08:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:49.269745 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d1e56-3f41-4132-8dc8-949dd9761a42 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:49.270168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d1e56-3f41-4132-8dc8-949dd9761a42 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:49.270361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d1e56-3f41-4132-8dc8-949dd9761a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:49.288350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-865d1e56-3f41-4132-8dc8-949dd9761a42 None None] MFA Rules not processed for user `022093dea10d46478b3016395c712d93`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:49.289538 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-865d1e56-3f41-4132-8dc8-949dd9761a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:49.298210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 678/1355] 10.222.0.65 () {58 vars in 1018 bytes} [Tue May 10 20:08:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:53.126777 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45d410-4f48-4317-ba4e-129af5e0f13c None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:53.127332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45d410-4f48-4317-ba4e-129af5e0f13c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:53.127332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45d410-4f48-4317-ba4e-129af5e0f13c None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:53.128670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 678/1356] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:53.133764 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cb30d-4c3d-4bf7-b476-ab2416e69b0e None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:53.134211 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cb30d-4c3d-4bf7-b476-ab2416e69b0e None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:53.134362 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cb30d-4c3d-4bf7-b476-ab2416e69b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:53.152555 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-f74cb30d-4c3d-4bf7-b476-ab2416e69b0e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:53.153786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f74cb30d-4c3d-4bf7-b476-ab2416e69b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:53.161947 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 679/1357] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:53.210010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72caa4ec-1b68-40f7-bac9-3d5bfab5f69e None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:53.210525 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72caa4ec-1b68-40f7-bac9-3d5bfab5f69e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:53.210950 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72caa4ec-1b68-40f7-bac9-3d5bfab5f69e None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:53.212570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 679/1358] 10.222.0.65 () {54 vars in 961 bytes} [Tue May 10 20:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:53.218498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029d7d0-3311-48d9-9f8a-d0efe6633c61 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:53.218900 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029d7d0-3311-48d9-9f8a-d0efe6633c61 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:53.219306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029d7d0-3311-48d9-9f8a-d0efe6633c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:53.272355 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-d029d7d0-3311-48d9-9f8a-d0efe6633c61 None None] MFA Rules not processed for user `ae922b2da4814d10bed1fa345700ac04`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:53.273705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-d029d7d0-3311-48d9-9f8a-d0efe6633c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:53.361733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 680/1359] 10.222.0.65 () {58 vars in 1058 bytes} [Tue May 10 20:08:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:53.369770 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72caa4ec-1b68-40f7-bac9-3d5bfab5f69e None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:08:53.385888 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72caa4ec-1b68-40f7-bac9-3d5bfab5f69e None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:08:53.391360 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398ad51-7064-48a3-8229-0a68c1ab1076 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:08:53.392002 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398ad51-7064-48a3-8229-0a68c1ab1076 service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:53.392284 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398ad51-7064-48a3-8229-0a68c1ab1076 service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:53.392744 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398ad51-7064-48a3-8229-0a68c1ab1076 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:53.400945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398ad51-7064-48a3-8229-0a68c1ab1076 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:08:53.402681 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398ad51-7064-48a3-8229-0a68c1ab1076 service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:08:53.412076 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 680/1360] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:08:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:08:55.881042 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cfd9d-f340-40f9-b19e-4b00d242a1c4 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:55.881538 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cfd9d-f340-40f9-b19e-4b00d242a1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:55.881538 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cfd9d-f340-40f9-b19e-4b00d242a1c4 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:55.882893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 681/1361] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:55.888764 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b32d75-fdfd-44d0-a23f-5c9759792b72 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:55.889182 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b32d75-fdfd-44d0-a23f-5c9759792b72 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:55.889312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b32d75-fdfd-44d0-a23f-5c9759792b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:55.908312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-65b32d75-fdfd-44d0-a23f-5c9759792b72 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:55.909499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-65b32d75-fdfd-44d0-a23f-5c9759792b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:55.917877 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 681/1362] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:55.961425 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cfd9d-f340-40f9-b19e-4b00d242a1c4 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:08:55.975414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cfd9d-f340-40f9-b19e-4b00d242a1c4 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:08:55.981029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d3a59-95b4-4285-a925-4a5bcddd85ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:08:55.981670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d3a59-95b4-4285-a925-4a5bcddd85ae service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:55.981952 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d3a59-95b4-4285-a925-4a5bcddd85ae service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:55.982257 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d3a59-95b4-4285-a925-4a5bcddd85ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:55.990502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d3a59-95b4-4285-a925-4a5bcddd85ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:08:55.992117 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d3a59-95b4-4285-a925-4a5bcddd85ae service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:08:56.002283 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 682/1363] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:08:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:08:58.696774 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fabb5-e993-4162-a0d2-839c0e0a9201 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:58.697260 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fabb5-e993-4162-a0d2-839c0e0a9201 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:58.697349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fabb5-e993-4162-a0d2-839c0e0a9201 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:58.698913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 682/1364] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:58.704199 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61e1b9-3b4d-4de2-9b99-f2ee28928434 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:58.704588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61e1b9-3b4d-4de2-9b99-f2ee28928434 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:58.704765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61e1b9-3b4d-4de2-9b99-f2ee28928434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:58.736064 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-ee61e1b9-3b4d-4de2-9b99-f2ee28928434 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:58.737287 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-ee61e1b9-3b4d-4de2-9b99-f2ee28928434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:58.746333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 683/1365] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:58.791977 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b027036-845d-4bbb-8c6a-4a6dac7b3cf1 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:58.792311 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b027036-845d-4bbb-8c6a-4a6dac7b3cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:58.792920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b027036-845d-4bbb-8c6a-4a6dac7b3cf1 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:58.793964 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 683/1366] 10.222.0.65 () {54 vars in 961 bytes} [Tue May 10 20:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:08:58.799550 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd3eec-761d-4eb4-a389-662d581d692a None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:58.799842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd3eec-761d-4eb4-a389-662d581d692a None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:58.800125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd3eec-761d-4eb4-a389-662d581d692a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:58.818009 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-bffd3eec-761d-4eb4-a389-662d581d692a None None] MFA Rules not processed for user `ae922b2da4814d10bed1fa345700ac04`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:08:58.819174 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-bffd3eec-761d-4eb4-a389-662d581d692a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:08:58.829595 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 684/1367] 10.222.0.65 () {58 vars in 1058 bytes} [Tue May 10 20:08:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:08:58.837505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b027036-845d-4bbb-8c6a-4a6dac7b3cf1 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:08:58.851946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b027036-845d-4bbb-8c6a-4a6dac7b3cf1 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:08:58.857689 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c0286-b2ff-43a6-aa61-1a76ae1cbe3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:08:58.858437 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c0286-b2ff-43a6-aa61-1a76ae1cbe3e service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:08:58.858769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c0286-b2ff-43a6-aa61-1a76ae1cbe3e service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:08:58.859041 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c0286-b2ff-43a6-aa61-1a76ae1cbe3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:08:58.867204 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c0286-b2ff-43a6-aa61-1a76ae1cbe3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:08:58.869394 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c0286-b2ff-43a6-aa61-1a76ae1cbe3e service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:08:58.878647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 684/1368] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:08:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:01.405321 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb105a3-a0c6-41a9-b1d9-e4d3e8bb2dc4 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:01.405898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb105a3-a0c6-41a9-b1d9-e4d3e8bb2dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:01.405898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb105a3-a0c6-41a9-b1d9-e4d3e8bb2dc4 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:01.407268 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 685/1369] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:01.412916 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dcb76-4778-4bc0-8316-4b380eb1c98c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:01.413354 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dcb76-4778-4bc0-8316-4b380eb1c98c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:01.413546 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dcb76-4778-4bc0-8316-4b380eb1c98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:01.437891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-4e7dcb76-4778-4bc0-8316-4b380eb1c98c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:01.439276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4e7dcb76-4778-4bc0-8316-4b380eb1c98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:01.447506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 685/1370] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:01.494635 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb105a3-a0c6-41a9-b1d9-e4d3e8bb2dc4 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:01.510239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb105a3-a0c6-41a9-b1d9-e4d3e8bb2dc4 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:01.516460 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7162402-d2b3-4093-a179-c4470c924da7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:01.517098 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7162402-d2b3-4093-a179-c4470c924da7 service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:01.517378 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7162402-d2b3-4093-a179-c4470c924da7 service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:01.517651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7162402-d2b3-4093-a179-c4470c924da7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:01.526842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7162402-d2b3-4093-a179-c4470c924da7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:01.528540 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7162402-d2b3-4093-a179-c4470c924da7 service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:01.529946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e2377-95a9-4857-946c-53294f49464a None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:01.530368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e2377-95a9-4857-946c-53294f49464a None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:01.530748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e2377-95a9-4857-946c-53294f49464a None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:01.532399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 686/1371] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:09:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 20:09:01.537460 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a01b3-6670-4a7a-a984-07032d3f0c05 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:01.537748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a01b3-6670-4a7a-a984-07032d3f0c05 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:01.538033 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a01b3-6670-4a7a-a984-07032d3f0c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:01.539705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 686/1372] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:01.558706 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-283a01b3-6670-4a7a-a984-07032d3f0c05 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:01.560014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-283a01b3-6670-4a7a-a984-07032d3f0c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:01.568417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 687/1373] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:09:01.892867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456f8ed-535a-499b-a055-c1b6920a558f None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:01.892867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456f8ed-535a-499b-a055-c1b6920a558f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:01.892867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456f8ed-535a-499b-a055-c1b6920a558f None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:01.893766 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 687/1374] 10.222.0.65 () {54 vars in 963 bytes} [Tue May 10 20:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:01.899084 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760d4fc-a6ba-4071-9875-5d23d55d9dd8 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:01.899361 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760d4fc-a6ba-4071-9875-5d23d55d9dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:01.899642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760d4fc-a6ba-4071-9875-5d23d55d9dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:01.953073 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-7760d4fc-a6ba-4071-9875-5d23d55d9dd8 None None] MFA Rules not processed for user `0e366ec240e5464d8c5e92f807123c23`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:01.954703 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7760d4fc-a6ba-4071-9875-5d23d55d9dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:02.013496 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 688/1375] 10.222.0.65 () {58 vars in 1060 bytes} [Tue May 10 20:09:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:02.021118 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456f8ed-535a-499b-a055-c1b6920a558f None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:02.031846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456f8ed-535a-499b-a055-c1b6920a558f None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:02.035854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad279a-742c-4bf0-b11b-286a4f518bcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:02.036693 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad279a-742c-4bf0-b11b-286a4f518bcf service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:02.037268 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad279a-742c-4bf0-b11b-286a4f518bcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:02.037702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad279a-742c-4bf0-b11b-286a4f518bcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:02.047356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad279a-742c-4bf0-b11b-286a4f518bcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:02.049004 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad279a-742c-4bf0-b11b-286a4f518bcf service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:02.058953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 688/1376] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:03.961879 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1090f-5dd6-4860-b319-009f32899d6b None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:03.962480 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1090f-5dd6-4860-b319-009f32899d6b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:03.962480 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1090f-5dd6-4860-b319-009f32899d6b None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:03.963902 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 689/1377] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:03.969741 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acdc3a-44ce-4e22-849f-b17e21204fa6 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:03.970235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acdc3a-44ce-4e22-849f-b17e21204fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:03.970358 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acdc3a-44ce-4e22-849f-b17e21204fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:03.988539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-d0acdc3a-44ce-4e22-849f-b17e21204fa6 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:03.989746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-d0acdc3a-44ce-4e22-849f-b17e21204fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:03.998839 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 689/1378] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:04.043949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1090f-5dd6-4860-b319-009f32899d6b None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:04.058744 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1090f-5dd6-4860-b319-009f32899d6b None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:04.064137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba1d07-056e-4de2-95b7-5570838930cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:04.064779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba1d07-056e-4de2-95b7-5570838930cc service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:04.065067 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba1d07-056e-4de2-95b7-5570838930cc service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:04.065463 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba1d07-056e-4de2-95b7-5570838930cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:04.074387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba1d07-056e-4de2-95b7-5570838930cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:04.075961 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba1d07-056e-4de2-95b7-5570838930cc service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:04.085714 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 690/1379] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:06.473718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb89fd-d4ff-4608-ab45-1a62c045d520 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:06.474602 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb89fd-d4ff-4608-ab45-1a62c045d520 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:06.474880 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb89fd-d4ff-4608-ab45-1a62c045d520 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:06.476235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 690/1380] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:09:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:06.481492 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893afcfc-14e8-422c-8164-12022fb77b9c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:06.481898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893afcfc-14e8-422c-8164-12022fb77b9c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:06.482045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893afcfc-14e8-422c-8164-12022fb77b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:06.500797 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-893afcfc-14e8-422c-8164-12022fb77b9c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:06.501966 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-893afcfc-14e8-422c-8164-12022fb77b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:06.510285 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 691/1381] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:06.553762 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb89fd-d4ff-4608-ab45-1a62c045d520 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:06.569717 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb89fd-d4ff-4608-ab45-1a62c045d520 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:06.575441 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009771ea-4244-41ae-a2c4-a9437b7ed8ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:06.576361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009771ea-4244-41ae-a2c4-a9437b7ed8ea service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:06.576642 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009771ea-4244-41ae-a2c4-a9437b7ed8ea service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:06.576914 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009771ea-4244-41ae-a2c4-a9437b7ed8ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:06.585193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009771ea-4244-41ae-a2c4-a9437b7ed8ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:06.586798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009771ea-4244-41ae-a2c4-a9437b7ed8ea service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:06.596852 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 691/1382] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:08.988344 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ada16-5aca-414b-888b-9b214d1f1b76 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:08.988922 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ada16-5aca-414b-888b-9b214d1f1b76 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:08.988922 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ada16-5aca-414b-888b-9b214d1f1b76 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:08.990289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 692/1383] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:09:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:08.996387 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603d1f9-c38f-4307-86d3-62a308219ca4 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:08.996797 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603d1f9-c38f-4307-86d3-62a308219ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:08.996947 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603d1f9-c38f-4307-86d3-62a308219ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:09.015087 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-d603d1f9-c38f-4307-86d3-62a308219ca4 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:09.016285 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-d603d1f9-c38f-4307-86d3-62a308219ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:09.024245 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 692/1384] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:09:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:09.066041 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ada16-5aca-414b-888b-9b214d1f1b76 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:09.081088 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ada16-5aca-414b-888b-9b214d1f1b76 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:09.086474 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad61d3-da22-4e9d-b16d-2403b692b90c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:09.087110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad61d3-da22-4e9d-b16d-2403b692b90c service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:09.087398 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad61d3-da22-4e9d-b16d-2403b692b90c service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:09.087679 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad61d3-da22-4e9d-b16d-2403b692b90c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:09.096569 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad61d3-da22-4e9d-b16d-2403b692b90c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:09.098179 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad61d3-da22-4e9d-b16d-2403b692b90c service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:09.108482 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 693/1385] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:11.774212 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badaa846-d27b-4a3b-9a32-f36a186cbd88 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:11.774913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badaa846-d27b-4a3b-9a32-f36a186cbd88 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:11.774913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badaa846-d27b-4a3b-9a32-f36a186cbd88 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:11.776653 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 693/1386] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:09:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:11.782760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcfd12-a788-4eee-9a08-12c438a946f8 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:11.783263 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcfd12-a788-4eee-9a08-12c438a946f8 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:11.783524 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcfd12-a788-4eee-9a08-12c438a946f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:11.803518 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-7adcfd12-a788-4eee-9a08-12c438a946f8 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:11.804782 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7adcfd12-a788-4eee-9a08-12c438a946f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:11.813241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 694/1387] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:09:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:11.860154 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badaa846-d27b-4a3b-9a32-f36a186cbd88 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:11.882323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badaa846-d27b-4a3b-9a32-f36a186cbd88 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:11.889899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24744f84-47cc-4192-9e32-09f4e585314b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:11.890602 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24744f84-47cc-4192-9e32-09f4e585314b service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:11.890889 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24744f84-47cc-4192-9e32-09f4e585314b service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:11.891170 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24744f84-47cc-4192-9e32-09f4e585314b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:11.899976 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24744f84-47cc-4192-9e32-09f4e585314b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:11.901567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24744f84-47cc-4192-9e32-09f4e585314b service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:11.911349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 694/1388] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:14.878389 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268849a1-bf6d-4c13-a945-9ea4cc29d94c None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:14.878928 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268849a1-bf6d-4c13-a945-9ea4cc29d94c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:14.879022 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268849a1-bf6d-4c13-a945-9ea4cc29d94c None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:14.880384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 695/1389] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:14.886620 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc1c59-36db-4a7c-a82d-6e3909e405bd None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:14.887043 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc1c59-36db-4a7c-a82d-6e3909e405bd None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:14.887184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc1c59-36db-4a7c-a82d-6e3909e405bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:14.905319 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-a4cc1c59-36db-4a7c-a82d-6e3909e405bd None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:14.906546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a4cc1c59-36db-4a7c-a82d-6e3909e405bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:14.914640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 695/1390] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:14.960362 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268849a1-bf6d-4c13-a945-9ea4cc29d94c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:14.975616 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268849a1-bf6d-4c13-a945-9ea4cc29d94c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:14.982066 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b78cb-fa7a-4657-a40e-bfbef94ec4d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:14.982742 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b78cb-fa7a-4657-a40e-bfbef94ec4d7 service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:14.983026 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b78cb-fa7a-4657-a40e-bfbef94ec4d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:14.983300 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b78cb-fa7a-4657-a40e-bfbef94ec4d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:14.991762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b78cb-fa7a-4657-a40e-bfbef94ec4d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:14.993324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b78cb-fa7a-4657-a40e-bfbef94ec4d7 service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:15.002744 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 696/1391] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:17.617312 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5411e6-9ec3-4be5-81b6-fe89fda72215 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:17.617906 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5411e6-9ec3-4be5-81b6-fe89fda72215 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:17.617906 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5411e6-9ec3-4be5-81b6-fe89fda72215 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:17.619302 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 696/1392] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:17.624411 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412557fc-79d3-401b-b453-5de91cd23041 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:17.624839 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412557fc-79d3-401b-b453-5de91cd23041 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:17.624970 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412557fc-79d3-401b-b453-5de91cd23041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:17.642562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-412557fc-79d3-401b-b453-5de91cd23041 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:17.643733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-412557fc-79d3-401b-b453-5de91cd23041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:17.651894 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 697/1393] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:17.694821 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5411e6-9ec3-4be5-81b6-fe89fda72215 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:17.710076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5411e6-9ec3-4be5-81b6-fe89fda72215 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:17.716089 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b86f7e-ad58-4ebe-bc82-be60d7e32454 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:17.716723 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b86f7e-ad58-4ebe-bc82-be60d7e32454 service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:17.717008 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b86f7e-ad58-4ebe-bc82-be60d7e32454 service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:17.717286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b86f7e-ad58-4ebe-bc82-be60d7e32454 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:17.725323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b86f7e-ad58-4ebe-bc82-be60d7e32454 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:17.726936 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b86f7e-ad58-4ebe-bc82-be60d7e32454 service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:17.736626 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 697/1394] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:18.350640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210a935-6fc5-4108-8d9d-612409328963 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:18.350854 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210a935-6fc5-4108-8d9d-612409328963 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:18.351129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210a935-6fc5-4108-8d9d-612409328963 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:18.352523 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 698/1395] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:18.358347 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689fbc6-e2f5-4daa-9030-a967a5fe52cf None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:18.358709 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689fbc6-e2f5-4daa-9030-a967a5fe52cf None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:18.358991 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689fbc6-e2f5-4daa-9030-a967a5fe52cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:18.377484 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-0689fbc6-e2f5-4daa-9030-a967a5fe52cf None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:18.378730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0689fbc6-e2f5-4daa-9030-a967a5fe52cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:18.387528 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 698/1396] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:09:18.447803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea56caf-44de-426b-9780-396c155190a9 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:18.448093 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea56caf-44de-426b-9780-396c155190a9 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:18.448374 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea56caf-44de-426b-9780-396c155190a9 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:18.449757 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 699/1397] 10.222.0.65 () {54 vars in 962 bytes} [Tue May 10 20:09:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:18.455333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64670ba8-7d47-478d-adad-5e8cf70d2541 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:18.455634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64670ba8-7d47-478d-adad-5e8cf70d2541 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:18.455920 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64670ba8-7d47-478d-adad-5e8cf70d2541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:18.504226 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-64670ba8-7d47-478d-adad-5e8cf70d2541 None None] MFA Rules not processed for user `67a252cde71a455fb1dc4bf9c47c0490`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:18.505428 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-64670ba8-7d47-478d-adad-5e8cf70d2541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:18.563674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 699/1398] 10.222.0.65 () {58 vars in 1059 bytes} [Tue May 10 20:09:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:09:18.571212 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea56caf-44de-426b-9780-396c155190a9 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:18.583010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea56caf-44de-426b-9780-396c155190a9 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:18.586592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68b598-4e73-4cc3-afc6-2b29897ffb03 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67a252cde71a455fb1dc4bf9c47c0490', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:18.587240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68b598-4e73-4cc3-afc6-2b29897ffb03 service glance] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:18.587519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68b598-4e73-4cc3-afc6-2b29897ffb03 service glance] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:18.588321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68b598-4e73-4cc3-afc6-2b29897ffb03 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:18.601018 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68b598-4e73-4cc3-afc6-2b29897ffb03 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:18.602456 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68b598-4e73-4cc3-afc6-2b29897ffb03 service glance] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:18.614447 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 700/1399] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:19.162004 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cd5c9-2aa0-40af-9a24-0912738cd60f None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:19.162556 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cd5c9-2aa0-40af-9a24-0912738cd60f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:19.162739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cd5c9-2aa0-40af-9a24-0912738cd60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:19.210197 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-f90cd5c9-2aa0-40af-9a24-0912738cd60f None None] MFA Rules not processed for user `7ec2605fd1b84e0ea134b943ed33a605`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:19.211400 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f90cd5c9-2aa0-40af-9a24-0912738cd60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:19.274107 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 700/1400] 10.222.0.65 () {58 vars in 1009 bytes} [Tue May 10 20:09:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:19.319647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d297aa-1668-4e48-ba67-a77b27bc1edf None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:19.320115 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d297aa-1668-4e48-ba67-a77b27bc1edf None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:19.320324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d297aa-1668-4e48-ba67-a77b27bc1edf None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:19.321771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 701/1401] 10.222.0.65 () {54 vars in 942 bytes} [Tue May 10 20:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 20:09:19.328243 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c65613-fd41-43d0-9bf9-f3213c82612b None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:19.328574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c65613-fd41-43d0-9bf9-f3213c82612b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:19.328857 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c65613-fd41-43d0-9bf9-f3213c82612b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:19.378004 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-06c65613-fd41-43d0-9bf9-f3213c82612b None None] MFA Rules not processed for user `b4d10b258a53484da22950e2b1db2e41`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:19.379238 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-06c65613-fd41-43d0-9bf9-f3213c82612b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:19.456080 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 701/1402] 10.222.0.65 () {58 vars in 1039 bytes} [Tue May 10 20:09:19 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:19.463408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d297aa-1668-4e48-ba67-a77b27bc1edf None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:19.476708 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d297aa-1668-4e48-ba67-a77b27bc1edf None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:19.481864 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e0978-ea30-47d4-a5cc-1e9dca58e2c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d10b258a53484da22950e2b1db2e41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:19.482543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e0978-ea30-47d4-a5cc-1e9dca58e2c1 service swift] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:19.482829 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e0978-ea30-47d4-a5cc-1e9dca58e2c1 service swift] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:19.483107 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e0978-ea30-47d4-a5cc-1e9dca58e2c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:19.491287 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e0978-ea30-47d4-a5cc-1e9dca58e2c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:19.492859 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e0978-ea30-47d4-a5cc-1e9dca58e2c1 service swift] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:19.501353 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 702/1403] 10.222.0.65 () {60 vars in 1384 bytes} [Tue May 10 20:09:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 20:09:20.105227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba505e-ad6b-4449-91e5-e396289aab46 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:20.105443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba505e-ad6b-4449-91e5-e396289aab46 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:20.105696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba505e-ad6b-4449-91e5-e396289aab46 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:20.110955 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 702/1404] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:09:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:20.112440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bc739-19bf-43d1-bff1-b6954fc6359a None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:20.112914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bc739-19bf-43d1-bff1-b6954fc6359a None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:20.113368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bc739-19bf-43d1-bff1-b6954fc6359a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:20.133384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-069bc739-19bf-43d1-bff1-b6954fc6359a None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:20.135294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-069bc739-19bf-43d1-bff1-b6954fc6359a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:20.147769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 703/1405] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:20.191500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba505e-ad6b-4449-91e5-e396289aab46 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:20.206759 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba505e-ad6b-4449-91e5-e396289aab46 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:20.212336 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb2412-1000-41a4-8705-794212caf61d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:20.213623 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb2412-1000-41a4-8705-794212caf61d service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:20.213904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb2412-1000-41a4-8705-794212caf61d service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:20.214241 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb2412-1000-41a4-8705-794212caf61d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:20.223010 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb2412-1000-41a4-8705-794212caf61d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:20.225737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb2412-1000-41a4-8705-794212caf61d service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:20.246460 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 703/1406] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:22.648397 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efa405-dc62-4359-a172-eaad49319336 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:22.648920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efa405-dc62-4359-a172-eaad49319336 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:22.648920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efa405-dc62-4359-a172-eaad49319336 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:22.652111 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 704/1407] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:09:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:22.656007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e62cdd-65b3-4d4a-8acb-8e7e29d13fa1 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:22.656434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e62cdd-65b3-4d4a-8acb-8e7e29d13fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:22.656560 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e62cdd-65b3-4d4a-8acb-8e7e29d13fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:22.675681 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-c7e62cdd-65b3-4d4a-8acb-8e7e29d13fa1 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:22.676123 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c7e62cdd-65b3-4d4a-8acb-8e7e29d13fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:22.684645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 704/1408] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:09:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:09:22.728816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efa405-dc62-4359-a172-eaad49319336 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:22.743211 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efa405-dc62-4359-a172-eaad49319336 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:22.749377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10675b-1347-4d48-8279-f8268d77d31f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:22.750009 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10675b-1347-4d48-8279-f8268d77d31f service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:22.750315 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10675b-1347-4d48-8279-f8268d77d31f service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:22.750587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10675b-1347-4d48-8279-f8268d77d31f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:22.758919 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10675b-1347-4d48-8279-f8268d77d31f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:22.761122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10675b-1347-4d48-8279-f8268d77d31f service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:22.770834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 705/1409] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:44.161934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b9d58-fa67-4294-8277-bddfd6de9cba None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:44.162572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b9d58-fa67-4294-8277-bddfd6de9cba None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:44.162572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b9d58-fa67-4294-8277-bddfd6de9cba None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:44.163820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 705/1410] 10.222.0.65 () {54 vars in 918 bytes} [Tue May 10 20:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:44.170317 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6b1e1-58be-4a41-bace-5cf45823441e None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:44.170762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6b1e1-58be-4a41-bace-5cf45823441e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:44.170871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6b1e1-58be-4a41-bace-5cf45823441e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:44.190410 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-d3b6b1e1-58be-4a41-bace-5cf45823441e None None] MFA Rules not processed for user `022093dea10d46478b3016395c712d93`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:44.191605 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-d3b6b1e1-58be-4a41-bace-5cf45823441e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:44.202673 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 706/1411] 10.222.0.65 () {58 vars in 1015 bytes} [Tue May 10 20:09:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:44.217157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7504bb-6a02-4394-9be8-8ddf0ad3401a None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:44.217451 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7504bb-6a02-4394-9be8-8ddf0ad3401a None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:44.217733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7504bb-6a02-4394-9be8-8ddf0ad3401a None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:44.219148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 706/1412] 10.222.0.65 () {54 vars in 918 bytes} [Tue May 10 20:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:44.224095 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff2d37-b36f-4f31-b091-c90becd9b838 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:44.224383 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff2d37-b36f-4f31-b091-c90becd9b838 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:44.224664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff2d37-b36f-4f31-b091-c90becd9b838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:44.242643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-6bff2d37-b36f-4f31-b091-c90becd9b838 None None] MFA Rules not processed for user `ae922b2da4814d10bed1fa345700ac04`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:44.244418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-6bff2d37-b36f-4f31-b091-c90becd9b838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:44.253590 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 707/1413] 10.222.0.65 () {58 vars in 1015 bytes} [Tue May 10 20:09:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:46.950174 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06328b-e741-49f9-8694-c5087b61f098 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:46.950739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06328b-e741-49f9-8694-c5087b61f098 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:46.950739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06328b-e741-49f9-8694-c5087b61f098 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:46.952078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 20:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:46.957292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9f97e-7c2e-45a4-b544-2904cb195ec6 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:46.957735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9f97e-7c2e-45a4-b544-2904cb195ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:46.957857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9f97e-7c2e-45a4-b544-2904cb195ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:46.976416 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-b2e9f97e-7c2e-45a4-b544-2904cb195ec6 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:46.978321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b2e9f97e-7c2e-45a4-b544-2904cb195ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:46.986073 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 20:09:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:09:47.026940 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06328b-e741-49f9-8694-c5087b61f098 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:47.041596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06328b-e741-49f9-8694-c5087b61f098 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:47.047023 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b40c6-f5eb-417d-b31a-33eddaa1d72a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:47.047656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b40c6-f5eb-417d-b31a-33eddaa1d72a service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:47.047941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b40c6-f5eb-417d-b31a-33eddaa1d72a service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:47.048219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b40c6-f5eb-417d-b31a-33eddaa1d72a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:47.056278 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b40c6-f5eb-417d-b31a-33eddaa1d72a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:47.057852 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b40c6-f5eb-417d-b31a-33eddaa1d72a service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:47.067045 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 708/1416] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:48.046485 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30897d-525c-4b8f-ac39-20d19f81b42e None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:48.047059 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30897d-525c-4b8f-ac39-20d19f81b42e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:48.047059 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30897d-525c-4b8f-ac39-20d19f81b42e None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:48.048410 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 20:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:48.053203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd3b75-d1ff-444a-8f50-746285f3ff3f None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:48.053622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd3b75-d1ff-444a-8f50-746285f3ff3f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:48.053758 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd3b75-d1ff-444a-8f50-746285f3ff3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:48.071875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-49bd3b75-d1ff-444a-8f50-746285f3ff3f None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:48.073005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-49bd3b75-d1ff-444a-8f50-746285f3ff3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:48.080951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 20:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:48.266525 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30897d-525c-4b8f-ac39-20d19f81b42e None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:48.281024 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30897d-525c-4b8f-ac39-20d19f81b42e None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:48.286330 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d45747-a023-4e8d-884b-5c8f032bb037 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:48.286957 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d45747-a023-4e8d-884b-5c8f032bb037 service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:48.287238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d45747-a023-4e8d-884b-5c8f032bb037 service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:48.287511 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d45747-a023-4e8d-884b-5c8f032bb037 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:48.295364 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d45747-a023-4e8d-884b-5c8f032bb037 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:48.296930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d45747-a023-4e8d-884b-5c8f032bb037 service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:48.306103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 710/1419] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:49.249710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e25cbe-9bee-44d9-bf7f-b0555be8261e None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:49.250299 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e25cbe-9bee-44d9-bf7f-b0555be8261e None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:49.250299 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e25cbe-9bee-44d9-bf7f-b0555be8261e None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:49.251649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 20:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:49.256637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ba63b-497e-4ff5-9c98-1ceb8c1f27ee None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:49.257022 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ba63b-497e-4ff5-9c98-1ceb8c1f27ee None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:49.257235 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ba63b-497e-4ff5-9c98-1ceb8c1f27ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:49.275510 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-d61ba63b-497e-4ff5-9c98-1ceb8c1f27ee None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:49.276693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-d61ba63b-497e-4ff5-9c98-1ceb8c1f27ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:49.284460 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 20:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:49.433147 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e25cbe-9bee-44d9-bf7f-b0555be8261e None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:49.447420 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e25cbe-9bee-44d9-bf7f-b0555be8261e None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:49.453056 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac681e68-1323-4ab4-9d45-f38c4446f666 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:49.453699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac681e68-1323-4ab4-9d45-f38c4446f666 service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:49.453984 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac681e68-1323-4ab4-9d45-f38c4446f666 service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:49.454298 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac681e68-1323-4ab4-9d45-f38c4446f666 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:49.462754 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac681e68-1323-4ab4-9d45-f38c4446f666 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:49.464330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac681e68-1323-4ab4-9d45-f38c4446f666 service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:49.473548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 711/1422] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:50.401273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676990a-3d7a-4397-9096-af9a68f211e5 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:50.401856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676990a-3d7a-4397-9096-af9a68f211e5 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:50.401856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676990a-3d7a-4397-9096-af9a68f211e5 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:50.403242 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 20:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:50.408181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf406a1-e5a3-4512-a0e5-19d0f34922c6 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:50.408625 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf406a1-e5a3-4512-a0e5-19d0f34922c6 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:50.408774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf406a1-e5a3-4512-a0e5-19d0f34922c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:50.426995 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-cbf406a1-e5a3-4512-a0e5-19d0f34922c6 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:50.428187 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-cbf406a1-e5a3-4512-a0e5-19d0f34922c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:50.436664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 20:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:50.583442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676990a-3d7a-4397-9096-af9a68f211e5 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:50.597286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676990a-3d7a-4397-9096-af9a68f211e5 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:50.603103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d3bb3-162e-4fec-a72f-43c461898322 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:50.603732 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d3bb3-162e-4fec-a72f-43c461898322 service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:50.604013 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d3bb3-162e-4fec-a72f-43c461898322 service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:50.604288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d3bb3-162e-4fec-a72f-43c461898322 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:50.612131 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d3bb3-162e-4fec-a72f-43c461898322 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:50.613690 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d3bb3-162e-4fec-a72f-43c461898322 service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:50.624384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 713/1425] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:51.553310 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e7f40-cb04-4509-8bbd-62a3a010e5da None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:51.553847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e7f40-cb04-4509-8bbd-62a3a010e5da None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:51.553980 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e7f40-cb04-4509-8bbd-62a3a010e5da None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:51.555786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 20:09:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:51.560667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e3607-3172-4897-ab8a-47a61e2d5a0c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:51.561053 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e3607-3172-4897-ab8a-47a61e2d5a0c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:51.561292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e3607-3172-4897-ab8a-47a61e2d5a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:51.579719 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-f37e3607-3172-4897-ab8a-47a61e2d5a0c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:51.580901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-f37e3607-3172-4897-ab8a-47a61e2d5a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:51.589024 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 20:09:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:51.737512 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e7f40-cb04-4509-8bbd-62a3a010e5da None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:51.751421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e7f40-cb04-4509-8bbd-62a3a010e5da None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:51.756650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e383c7-10e8-4aec-9ee8-a32f1d8b7576 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:51.757271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e383c7-10e8-4aec-9ee8-a32f1d8b7576 service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:51.757549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e383c7-10e8-4aec-9ee8-a32f1d8b7576 service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:51.757820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e383c7-10e8-4aec-9ee8-a32f1d8b7576 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:51.766216 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e383c7-10e8-4aec-9ee8-a32f1d8b7576 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:51.767779 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e383c7-10e8-4aec-9ee8-a32f1d8b7576 service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:51.777739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 714/1428] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:52.750179 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae63f21-3726-4760-a281-3ce9a40933d8 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:52.750739 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae63f21-3726-4760-a281-3ce9a40933d8 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:52.750926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae63f21-3726-4760-a281-3ce9a40933d8 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:52.753161 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 20:09:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:52.758592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be69e3f-6c8a-459a-9125-8e7de7689ead None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:52.759041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be69e3f-6c8a-459a-9125-8e7de7689ead None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:52.759167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be69e3f-6c8a-459a-9125-8e7de7689ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:52.777483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-1be69e3f-6c8a-459a-9125-8e7de7689ead None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:52.778746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1be69e3f-6c8a-459a-9125-8e7de7689ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:52.786458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 20:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:52.941778 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae63f21-3726-4760-a281-3ce9a40933d8 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:52.957299 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae63f21-3726-4760-a281-3ce9a40933d8 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:52.962613 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2b57f-f49b-4e3f-8630-372cc370e37b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:52.963243 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2b57f-f49b-4e3f-8630-372cc370e37b service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:52.963530 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2b57f-f49b-4e3f-8630-372cc370e37b service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:52.963824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2b57f-f49b-4e3f-8630-372cc370e37b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:52.971724 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2b57f-f49b-4e3f-8630-372cc370e37b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:52.973289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2b57f-f49b-4e3f-8630-372cc370e37b service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:52.982324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 716/1431] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:53.929421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f038e75-566f-421d-9c26-bf79892d0175 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:53.930041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f038e75-566f-421d-9c26-bf79892d0175 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:53.930041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f038e75-566f-421d-9c26-bf79892d0175 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:53.931448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 20:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:53.936576 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c007b4-a7f5-4844-8b77-8ed6e81ea9c7 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:53.936994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c007b4-a7f5-4844-8b77-8ed6e81ea9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:53.937154 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c007b4-a7f5-4844-8b77-8ed6e81ea9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:53.954959 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-96c007b4-a7f5-4844-8b77-8ed6e81ea9c7 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:53.956129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-96c007b4-a7f5-4844-8b77-8ed6e81ea9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:53.964214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 20:09:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:09:54.112706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f038e75-566f-421d-9c26-bf79892d0175 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:54.127555 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f038e75-566f-421d-9c26-bf79892d0175 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:54.133459 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf15473-2e57-450c-b44f-72dc3afd0641 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:54.134113 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf15473-2e57-450c-b44f-72dc3afd0641 service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:54.134399 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf15473-2e57-450c-b44f-72dc3afd0641 service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:54.134677 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf15473-2e57-450c-b44f-72dc3afd0641 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:54.142614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf15473-2e57-450c-b44f-72dc3afd0641 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:54.144177 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf15473-2e57-450c-b44f-72dc3afd0641 service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:54.153931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 717/1434] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:55.133531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a60ae7-4288-4c6c-982a-dd82d7905624 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:55.134205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a60ae7-4288-4c6c-982a-dd82d7905624 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:55.134205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a60ae7-4288-4c6c-982a-dd82d7905624 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:55.135491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 20:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:55.140537 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a1ce7-22ba-4e3f-8b00-9bbee3594d0c None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:55.140952 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a1ce7-22ba-4e3f-8b00-9bbee3594d0c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:55.141098 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a1ce7-22ba-4e3f-8b00-9bbee3594d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:55.158959 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-f60a1ce7-22ba-4e3f-8b00-9bbee3594d0c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:55.160134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f60a1ce7-22ba-4e3f-8b00-9bbee3594d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:55.168465 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 20:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:55.326467 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a60ae7-4288-4c6c-982a-dd82d7905624 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:55.341196 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a60ae7-4288-4c6c-982a-dd82d7905624 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:55.346510 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9989b-b1e2-4ff9-a092-278079ee9d7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:55.347141 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9989b-b1e2-4ff9-a092-278079ee9d7b service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:55.347421 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9989b-b1e2-4ff9-a092-278079ee9d7b service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:55.347699 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9989b-b1e2-4ff9-a092-278079ee9d7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:55.356227 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9989b-b1e2-4ff9-a092-278079ee9d7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:55.357831 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9989b-b1e2-4ff9-a092-278079ee9d7b service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:55.367022 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 719/1437] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:56.300939 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9a29c-30d0-4a16-9a7d-e80d220a67ac None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:56.301506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9a29c-30d0-4a16-9a7d-e80d220a67ac None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:56.301506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9a29c-30d0-4a16-9a7d-e80d220a67ac None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:56.302905 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 20:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:56.308368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc012f48-0ada-474b-bfe4-be2d690e8fe3 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:56.308770 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc012f48-0ada-474b-bfe4-be2d690e8fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:56.308927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc012f48-0ada-474b-bfe4-be2d690e8fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:56.326922 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-cc012f48-0ada-474b-bfe4-be2d690e8fe3 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:56.328101 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-cc012f48-0ada-474b-bfe4-be2d690e8fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:56.336023 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 20:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:56.484518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9a29c-30d0-4a16-9a7d-e80d220a67ac None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:56.498958 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9a29c-30d0-4a16-9a7d-e80d220a67ac None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:56.504323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b9250-e6bf-454d-9db3-a2defe96dfe0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:56.504948 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b9250-e6bf-454d-9db3-a2defe96dfe0 service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:56.505231 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b9250-e6bf-454d-9db3-a2defe96dfe0 service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:56.505509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b9250-e6bf-454d-9db3-a2defe96dfe0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:56.513345 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b9250-e6bf-454d-9db3-a2defe96dfe0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:56.514944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b9250-e6bf-454d-9db3-a2defe96dfe0 service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:56.524014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 720/1440] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:09:58.596624 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778b063-bf93-4296-99f6-3e0498d1204d None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:58.597183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778b063-bf93-4296-99f6-3e0498d1204d None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:58.597183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778b063-bf93-4296-99f6-3e0498d1204d None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:58.598973 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 20:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:09:58.604813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac2111-91d3-4eb2-b541-0dcae81e1902 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:58.605202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac2111-91d3-4eb2-b541-0dcae81e1902 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:58.605371 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac2111-91d3-4eb2-b541-0dcae81e1902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:58.623271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-3bac2111-91d3-4eb2-b541-0dcae81e1902 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:09:58.624468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-3bac2111-91d3-4eb2-b541-0dcae81e1902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:09:58.632592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 20:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:09:58.675063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778b063-bf93-4296-99f6-3e0498d1204d None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:09:58.686212 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778b063-bf93-4296-99f6-3e0498d1204d None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:09:58.689619 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a2edc-6152-463b-982b-72b9c70338b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:09:58.690278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a2edc-6152-463b-982b-72b9c70338b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:09:58.690560 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a2edc-6152-463b-982b-72b9c70338b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:09:58.690841 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a2edc-6152-463b-982b-72b9c70338b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:09:58.699741 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a2edc-6152-463b-982b-72b9c70338b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:09:58.701153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a2edc-6152-463b-982b-72b9c70338b4 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:09:58.710242 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 722/1443] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:09:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:10:01.081658 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52f48a-d8e0-47c8-a025-c50cd0e93927 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:01.082225 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52f48a-d8e0-47c8-a025-c50cd0e93927 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:01.082225 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52f48a-d8e0-47c8-a025-c50cd0e93927 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:01.083647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 20:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:10:01.089581 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece69b8-8a3f-4be6-9c8f-3a5458b24588 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:01.089990 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece69b8-8a3f-4be6-9c8f-3a5458b24588 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:01.090168 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece69b8-8a3f-4be6-9c8f-3a5458b24588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:01.107308 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-aece69b8-8a3f-4be6-9c8f-3a5458b24588 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:10:01.108499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-aece69b8-8a3f-4be6-9c8f-3a5458b24588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:10:01.116209 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 20:10:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:10:01.154713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52f48a-d8e0-47c8-a025-c50cd0e93927 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:10:01.165172 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52f48a-d8e0-47c8-a025-c50cd0e93927 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:10:01.168705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98af620-e736-4e4a-bbcf-bb4087cc7144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:10:01.169333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98af620-e736-4e4a-bbcf-bb4087cc7144 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:01.169616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98af620-e736-4e4a-bbcf-bb4087cc7144 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:01.169892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98af620-e736-4e4a-bbcf-bb4087cc7144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:01.178756 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98af620-e736-4e4a-bbcf-bb4087cc7144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:10:01.180798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98af620-e736-4e4a-bbcf-bb4087cc7144 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:10:01.190309 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 723/1446] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:10:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:10:03.836815 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a18fd1-316c-4abd-9b99-6ddff3f16a6a None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:03.837322 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a18fd1-316c-4abd-9b99-6ddff3f16a6a None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:03.837430 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a18fd1-316c-4abd-9b99-6ddff3f16a6a None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:03.838846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 20:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:10:03.844073 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bc8e2-607f-4217-b33e-52696ac3cd6f None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:03.844488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bc8e2-607f-4217-b33e-52696ac3cd6f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:03.844637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bc8e2-607f-4217-b33e-52696ac3cd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:03.862616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-442bc8e2-607f-4217-b33e-52696ac3cd6f None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:10:03.863793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-442bc8e2-607f-4217-b33e-52696ac3cd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:10:03.872114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 20:10:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:10:03.908931 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a18fd1-316c-4abd-9b99-6ddff3f16a6a None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:10:03.920634 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a18fd1-316c-4abd-9b99-6ddff3f16a6a None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:10:03.924261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e392af-da1c-4f8b-8c87-873b5f425003 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:10:03.924891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e392af-da1c-4f8b-8c87-873b5f425003 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:03.925172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e392af-da1c-4f8b-8c87-873b5f425003 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:03.925445 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e392af-da1c-4f8b-8c87-873b5f425003 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:03.935520 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e392af-da1c-4f8b-8c87-873b5f425003 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:10:03.936999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e392af-da1c-4f8b-8c87-873b5f425003 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:10:03.946632 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 725/1449] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:10:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:10:06.296755 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29495ad0-d168-4a05-aa9b-9e728477bf8b None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:06.297323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29495ad0-d168-4a05-aa9b-9e728477bf8b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:06.297323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29495ad0-d168-4a05-aa9b-9e728477bf8b None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:06.298745 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 20:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:10:06.304155 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a100095-88c8-40c1-8a39-6f802291edf3 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:06.304597 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a100095-88c8-40c1-8a39-6f802291edf3 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:06.304704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a100095-88c8-40c1-8a39-6f802291edf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:06.322700 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-0a100095-88c8-40c1-8a39-6f802291edf3 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:10:06.323889 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-0a100095-88c8-40c1-8a39-6f802291edf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:10:06.331717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 20:10:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:10:06.369810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29495ad0-d168-4a05-aa9b-9e728477bf8b None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:10:06.380436 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29495ad0-d168-4a05-aa9b-9e728477bf8b None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:10:06.384549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80700a25-39a7-4f9a-8a25-18c36c2bcf32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:10:06.385180 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80700a25-39a7-4f9a-8a25-18c36c2bcf32 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:06.385465 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80700a25-39a7-4f9a-8a25-18c36c2bcf32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:06.385738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80700a25-39a7-4f9a-8a25-18c36c2bcf32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:06.394327 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80700a25-39a7-4f9a-8a25-18c36c2bcf32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:10:06.395741 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80700a25-39a7-4f9a-8a25-18c36c2bcf32 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:10:06.404950 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 726/1452] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:10:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:10:08.660807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5901516-293c-40ae-91e3-22150b25d78c None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:08.661367 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5901516-293c-40ae-91e3-22150b25d78c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:08.661367 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5901516-293c-40ae-91e3-22150b25d78c None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:08.662763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 20:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:10:08.668671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fb583-dbd7-4bbf-8010-92c8bccbcc59 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:08.669101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fb583-dbd7-4bbf-8010-92c8bccbcc59 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:08.669258 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fb583-dbd7-4bbf-8010-92c8bccbcc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:08.686665 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-965fb583-dbd7-4bbf-8010-92c8bccbcc59 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:10:08.687861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-965fb583-dbd7-4bbf-8010-92c8bccbcc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:10:08.695785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 20:10:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:10:08.733766 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5901516-293c-40ae-91e3-22150b25d78c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:10:08.744223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5901516-293c-40ae-91e3-22150b25d78c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:10:08.747734 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6404f80-a63e-4141-9482-ac9d24cd959b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:10:08.748368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6404f80-a63e-4141-9482-ac9d24cd959b service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:08.748654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6404f80-a63e-4141-9482-ac9d24cd959b service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:08.748930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6404f80-a63e-4141-9482-ac9d24cd959b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:08.757659 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6404f80-a63e-4141-9482-ac9d24cd959b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:10:08.759116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6404f80-a63e-4141-9482-ac9d24cd959b service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:10:08.768251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 728/1455] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:10:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:10:11.068739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85438e09-d224-43f9-ad2c-0d29a473d272 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:11.069322 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85438e09-d224-43f9-ad2c-0d29a473d272 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:11.069322 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85438e09-d224-43f9-ad2c-0d29a473d272 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:11.070829 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 20:10:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:10:11.076726 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed6ef8-1d2d-4642-b800-7ddf368acc7e None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:11.077132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed6ef8-1d2d-4642-b800-7ddf368acc7e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:11.077287 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed6ef8-1d2d-4642-b800-7ddf368acc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:11.095257 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-69ed6ef8-1d2d-4642-b800-7ddf368acc7e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:10:11.096442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-69ed6ef8-1d2d-4642-b800-7ddf368acc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:10:11.105036 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 20:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:10:11.141370 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85438e09-d224-43f9-ad2c-0d29a473d272 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:10:11.152545 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85438e09-d224-43f9-ad2c-0d29a473d272 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:10:11.156069 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d472a-21bb-44c6-8580-cd8d321d790d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:10:11.156700 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d472a-21bb-44c6-8580-cd8d321d790d service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:11.156987 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d472a-21bb-44c6-8580-cd8d321d790d service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:11.157266 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d472a-21bb-44c6-8580-cd8d321d790d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:11.166645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d472a-21bb-44c6-8580-cd8d321d790d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:10:11.168071 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d472a-21bb-44c6-8580-cd8d321d790d service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:10:11.177224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 729/1458] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:10:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:10:13.436672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afb152-f821-4ba2-866c-9f98393f3bc5 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:13.437170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afb152-f821-4ba2-866c-9f98393f3bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:13.437305 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afb152-f821-4ba2-866c-9f98393f3bc5 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:13.438739 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 20:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:10:13.443963 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61c418-d0e9-40cb-b06b-9d593c3668a9 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:13.444381 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61c418-d0e9-40cb-b06b-9d593c3668a9 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:13.444561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61c418-d0e9-40cb-b06b-9d593c3668a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:13.462716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-9d61c418-d0e9-40cb-b06b-9d593c3668a9 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:10:13.463901 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-9d61c418-d0e9-40cb-b06b-9d593c3668a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:10:13.471697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 20:10:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:10:13.508371 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70afb152-f821-4ba2-866c-9f98393f3bc5 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:10:13.518681 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70afb152-f821-4ba2-866c-9f98393f3bc5 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:10:13.522193 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b04d2c-875c-4f7c-bda9-a75d41d9c769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:10:13.522926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b04d2c-875c-4f7c-bda9-a75d41d9c769 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:13.523658 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b04d2c-875c-4f7c-bda9-a75d41d9c769 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:13.523932 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b04d2c-875c-4f7c-bda9-a75d41d9c769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:13.532727 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b04d2c-875c-4f7c-bda9-a75d41d9c769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:10:13.534304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b04d2c-875c-4f7c-bda9-a75d41d9c769 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:10:13.543703 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 731/1461] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:10:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:10:15.817642 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ed015-4b57-4d3c-847a-832622cf9bfb None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:15.818232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ed015-4b57-4d3c-847a-832622cf9bfb None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:15.818232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ed015-4b57-4d3c-847a-832622cf9bfb None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:15.820153 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 20:10:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:10:15.825590 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb35601-d7d2-4ea7-82b0-bb8f413b1773 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:15.825971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb35601-d7d2-4ea7-82b0-bb8f413b1773 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:15.826188 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb35601-d7d2-4ea7-82b0-bb8f413b1773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:15.846301 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-3fb35601-d7d2-4ea7-82b0-bb8f413b1773 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:10:15.847501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3fb35601-d7d2-4ea7-82b0-bb8f413b1773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:10:15.855624 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 20:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:10:15.891740 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ed015-4b57-4d3c-847a-832622cf9bfb None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:10:15.902692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ed015-4b57-4d3c-847a-832622cf9bfb None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:10:15.906156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580edc4f-10d7-442b-948f-063d881271d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:10:15.906784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580edc4f-10d7-442b-948f-063d881271d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:15.907064 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580edc4f-10d7-442b-948f-063d881271d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:15.907337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580edc4f-10d7-442b-948f-063d881271d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:15.915888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580edc4f-10d7-442b-948f-063d881271d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:10:15.917302 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580edc4f-10d7-442b-948f-063d881271d0 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:10:15.926332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 732/1464] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:10:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:10:18.140799 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a5c6a-8a4f-44bf-bd7c-615fde7a0c8c None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:18.141312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a5c6a-8a4f-44bf-bd7c-615fde7a0c8c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:18.141312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a5c6a-8a4f-44bf-bd7c-615fde7a0c8c None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:18.142720 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 20:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:10:18.148508 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c38485-91fb-4fd9-8fe4-ea29549f7d34 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:18.148905 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c38485-91fb-4fd9-8fe4-ea29549f7d34 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:18.149056 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c38485-91fb-4fd9-8fe4-ea29549f7d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:18.166897 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-55c38485-91fb-4fd9-8fe4-ea29549f7d34 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:10:18.168074 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-55c38485-91fb-4fd9-8fe4-ea29549f7d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:10:18.175784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 20:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:10:18.212235 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a5c6a-8a4f-44bf-bd7c-615fde7a0c8c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:10:18.222714 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a5c6a-8a4f-44bf-bd7c-615fde7a0c8c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:10:18.226274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52097d05-9f17-413b-b10e-33625c9d3d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:10:18.226934 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52097d05-9f17-413b-b10e-33625c9d3d2e service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:18.227216 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52097d05-9f17-413b-b10e-33625c9d3d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:18.227498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52097d05-9f17-413b-b10e-33625c9d3d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:18.236141 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52097d05-9f17-413b-b10e-33625c9d3d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:10:18.237546 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52097d05-9f17-413b-b10e-33625c9d3d2e service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:10:18.246638 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 734/1467] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:10:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:10:20.500650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e338606-a181-427b-8f76-50f7d1465e7c None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:20.501219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e338606-a181-427b-8f76-50f7d1465e7c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:20.501219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e338606-a181-427b-8f76-50f7d1465e7c None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:20.502643 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 20:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:10:20.508742 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b4305-7dff-4c6a-9af2-81106e1a629b None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:20.509186 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b4305-7dff-4c6a-9af2-81106e1a629b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:20.509303 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b4305-7dff-4c6a-9af2-81106e1a629b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:20.527431 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-573b4305-7dff-4c6a-9af2-81106e1a629b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:10:20.528628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-573b4305-7dff-4c6a-9af2-81106e1a629b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:10:20.536663 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 20:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:10:20.573054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e338606-a181-427b-8f76-50f7d1465e7c None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:10:20.584133 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e338606-a181-427b-8f76-50f7d1465e7c None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:10:20.588337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a2411-8c64-4899-b484-ccbea93bbdfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:10:20.588978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a2411-8c64-4899-b484-ccbea93bbdfb service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:20.589264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a2411-8c64-4899-b484-ccbea93bbdfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:20.589541 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a2411-8c64-4899-b484-ccbea93bbdfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:20.598628 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a2411-8c64-4899-b484-ccbea93bbdfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:10:20.600065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a2411-8c64-4899-b484-ccbea93bbdfb service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:10:20.609864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 735/1470] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:10:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:10:54.309437 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f2c34-e24f-4d82-9bc9-28baf94da6b4 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:54.310076 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f2c34-e24f-4d82-9bc9-28baf94da6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:54.310076 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f2c34-e24f-4d82-9bc9-28baf94da6b4 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:54.311450 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 736/1471] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:10:54.316578 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f43363-5595-46fb-be24-c01b32f7a035 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:54.322232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f43363-5595-46fb-be24-c01b32f7a035 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:54.322232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f43363-5595-46fb-be24-c01b32f7a035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:54.335976 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-49f43363-5595-46fb-be24-c01b32f7a035 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:10:54.337158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-49f43363-5595-46fb-be24-c01b32f7a035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:10:54.345218 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 736/1472] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:10:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:10:54.391730 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca953c-ffe8-405c-9791-780288bbcc84 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:54.392025 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca953c-ffe8-405c-9791-780288bbcc84 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:54.392310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca953c-ffe8-405c-9791-780288bbcc84 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:54.393654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 737/1473] 10.222.0.65 () {54 vars in 962 bytes} [Tue May 10 20:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:10:54.398321 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65b678-8a77-4cb4-8b1f-1502125d16ca None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:54.398617 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65b678-8a77-4cb4-8b1f-1502125d16ca None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:54.398902 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65b678-8a77-4cb4-8b1f-1502125d16ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:54.416322 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-5d65b678-8a77-4cb4-8b1f-1502125d16ca None None] MFA Rules not processed for user `67a252cde71a455fb1dc4bf9c47c0490`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:10:54.417515 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-5d65b678-8a77-4cb4-8b1f-1502125d16ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:10:54.424162 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 737/1474] 10.222.0.65 () {58 vars in 1059 bytes} [Tue May 10 20:10:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:10:54.432188 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca953c-ffe8-405c-9791-780288bbcc84 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:10:54.443162 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca953c-ffe8-405c-9791-780288bbcc84 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:10:54.447012 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd0ad5-bd48-4cd5-b36b-7b6a7852cefc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67a252cde71a455fb1dc4bf9c47c0490', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:10:54.447641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd0ad5-bd48-4cd5-b36b-7b6a7852cefc service glance] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:54.447927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd0ad5-bd48-4cd5-b36b-7b6a7852cefc service glance] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:54.448204 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd0ad5-bd48-4cd5-b36b-7b6a7852cefc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:54.456938 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd0ad5-bd48-4cd5-b36b-7b6a7852cefc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:10:54.458392 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd0ad5-bd48-4cd5-b36b-7b6a7852cefc service glance] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:10:54.467449 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 738/1475] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:10:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:10:56.773372 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d8f91-877c-4ffb-89f0-f49309111bf1 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:56.773881 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d8f91-877c-4ffb-89f0-f49309111bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:56.773881 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d8f91-877c-4ffb-89f0-f49309111bf1 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:56.775298 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 738/1476] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:10:56.780987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab241a-f17f-408b-89bc-16a6f5bf2af1 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:56.781438 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab241a-f17f-408b-89bc-16a6f5bf2af1 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:56.781544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab241a-f17f-408b-89bc-16a6f5bf2af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:56.799279 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-39ab241a-f17f-408b-89bc-16a6f5bf2af1 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:10:56.800466 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-39ab241a-f17f-408b-89bc-16a6f5bf2af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:10:56.808264 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 739/1477] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:10:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:10:56.831192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d8f91-877c-4ffb-89f0-f49309111bf1 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:10:56.844109 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d8f91-877c-4ffb-89f0-f49309111bf1 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:10:56.849117 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccd8bc-851f-4c23-adfe-41e875291820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:10:56.849753 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccd8bc-851f-4c23-adfe-41e875291820 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:56.850035 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccd8bc-851f-4c23-adfe-41e875291820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:56.850340 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccd8bc-851f-4c23-adfe-41e875291820 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:56.851074 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ccd8bc-851f-4c23-adfe-41e875291820 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:10:56.852871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: WARNING py.warnings [None req-c8ccd8bc-851f-4c23-adfe-41e875291820 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 20:10:56.852871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: warnings.warn(msg) May 10 20:10:56.852871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: May 10 20:10:56.854030 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ccd8bc-851f-4c23-adfe-41e875291820 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:10:56.872585 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 739/1478] 10.222.0.65 () {56 vars in 1108 bytes} [Tue May 10 20:10:56 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:10:59.124703 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdad36-c526-4ec7-bde8-d843f7bbd9e4 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:59.125282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdad36-c526-4ec7-bde8-d843f7bbd9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:59.125282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdad36-c526-4ec7-bde8-d843f7bbd9e4 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:59.126786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 740/1479] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:10:59.132008 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f8c50-3da0-42b6-83af-a90134699008 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:59.132464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f8c50-3da0-42b6-83af-a90134699008 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:59.132601 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f8c50-3da0-42b6-83af-a90134699008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:59.150518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-4d0f8c50-3da0-42b6-83af-a90134699008 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:10:59.151730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-4d0f8c50-3da0-42b6-83af-a90134699008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:10:59.159527 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 740/1480] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:10:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:10:59.201074 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdad36-c526-4ec7-bde8-d843f7bbd9e4 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:10:59.215449 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdad36-c526-4ec7-bde8-d843f7bbd9e4 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:10:59.220672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63979aec-2cdc-4196-ab18-fa7b93fa3876 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:10:59.221296 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63979aec-2cdc-4196-ab18-fa7b93fa3876 service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:10:59.221576 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63979aec-2cdc-4196-ab18-fa7b93fa3876 service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:10:59.221848 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63979aec-2cdc-4196-ab18-fa7b93fa3876 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:10:59.230388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63979aec-2cdc-4196-ab18-fa7b93fa3876 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:10:59.232568 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63979aec-2cdc-4196-ab18-fa7b93fa3876 service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:10:59.241693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 741/1481] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:10:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:11:01.657825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc1c7e-312c-42ad-9d23-35e949cdc51f None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:11:01.658421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc1c7e-312c-42ad-9d23-35e949cdc51f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:11:01.658421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc1c7e-312c-42ad-9d23-35e949cdc51f None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:11:01.659790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 741/1482] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 20:11:01.665088 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4896d3-8860-4f87-bcd0-5e5cdb855046 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:11:01.665494 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4896d3-8860-4f87-bcd0-5e5cdb855046 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:11:01.665633 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4896d3-8860-4f87-bcd0-5e5cdb855046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:11:01.682831 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-2a4896d3-8860-4f87-bcd0-5e5cdb855046 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:11:01.683969 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-2a4896d3-8860-4f87-bcd0-5e5cdb855046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:11:01.692205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 742/1483] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:11:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:11:01.728099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc1c7e-312c-42ad-9d23-35e949cdc51f None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:11:01.738672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc1c7e-312c-42ad-9d23-35e949cdc51f None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:11:01.742224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1711a8-b7aa-4624-973b-b2e259b2d810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:11:01.742862 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1711a8-b7aa-4624-973b-b2e259b2d810 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:11:01.743150 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1711a8-b7aa-4624-973b-b2e259b2d810 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:11:01.743440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1711a8-b7aa-4624-973b-b2e259b2d810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:11:01.753231 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1711a8-b7aa-4624-973b-b2e259b2d810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:11:01.755279 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1711a8-b7aa-4624-973b-b2e259b2d810 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:11:01.765284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 742/1484] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:11:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:11:04.061471 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ab47f-1507-48d8-80d0-627317169fed None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:11:04.062024 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ab47f-1507-48d8-80d0-627317169fed None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:11:04.062024 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ab47f-1507-48d8-80d0-627317169fed None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:11:04.063402 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 743/1485] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:11:04.069391 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885d923-9d24-445a-8985-f845fb92a971 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:11:04.069818 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885d923-9d24-445a-8985-f845fb92a971 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:11:04.069989 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885d923-9d24-445a-8985-f845fb92a971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:11:04.087826 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-3885d923-9d24-445a-8985-f845fb92a971 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:11:04.089020 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-3885d923-9d24-445a-8985-f845fb92a971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:11:04.096838 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 743/1486] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:11:04.134473 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ab47f-1507-48d8-80d0-627317169fed None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:11:04.144848 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ab47f-1507-48d8-80d0-627317169fed None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:11:04.148438 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2441e-d222-482d-866e-280b08cff0fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:11:04.149057 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2441e-d222-482d-866e-280b08cff0fc service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:11:04.149337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2441e-d222-482d-866e-280b08cff0fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:11:04.149607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2441e-d222-482d-866e-280b08cff0fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:11:04.158354 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2441e-d222-482d-866e-280b08cff0fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:11:04.159760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2441e-d222-482d-866e-280b08cff0fc service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:11:04.168996 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 744/1487] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:11:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:11:06.561043 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c49ce-c8a2-41d9-9dc4-508439fbe859 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:11:06.561646 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c49ce-c8a2-41d9-9dc4-508439fbe859 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:11:06.561646 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c49ce-c8a2-41d9-9dc4-508439fbe859 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:11:06.563044 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 744/1488] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:11:06.569178 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d14c2e-d588-4eae-9895-93be833601e3 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:11:06.569585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d14c2e-d588-4eae-9895-93be833601e3 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:11:06.569768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d14c2e-d588-4eae-9895-93be833601e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:11:06.587475 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-41d14c2e-d588-4eae-9895-93be833601e3 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:11:06.588656 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-41d14c2e-d588-4eae-9895-93be833601e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:11:06.596273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 745/1489] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:11:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:11:06.636077 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c49ce-c8a2-41d9-9dc4-508439fbe859 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:11:06.646467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c49ce-c8a2-41d9-9dc4-508439fbe859 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:11:06.649915 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7750d-ab55-41fc-a652-f1dc3f6e7013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:11:06.650582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7750d-ab55-41fc-a652-f1dc3f6e7013 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:11:06.650867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7750d-ab55-41fc-a652-f1dc3f6e7013 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:11:06.651143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7750d-ab55-41fc-a652-f1dc3f6e7013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:11:06.659745 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7750d-ab55-41fc-a652-f1dc3f6e7013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:11:06.661177 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7750d-ab55-41fc-a652-f1dc3f6e7013 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:11:06.670324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 745/1490] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:11:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:11:09.161305 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed610f2-271e-4134-b14b-61d6db24387e None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:11:09.161851 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed610f2-271e-4134-b14b-61d6db24387e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:11:09.161851 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed610f2-271e-4134-b14b-61d6db24387e None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:11:09.163286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 746/1491] 10.222.0.65 () {54 vars in 926 bytes} [Tue May 10 20:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:11:09.168411 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3576685-8b63-45aa-9644-5f03c8ca2990 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:11:09.168819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3576685-8b63-45aa-9644-5f03c8ca2990 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:11:09.168963 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3576685-8b63-45aa-9644-5f03c8ca2990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:11:09.186323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-e3576685-8b63-45aa-9644-5f03c8ca2990 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:11:09.187498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-e3576685-8b63-45aa-9644-5f03c8ca2990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:11:09.196257 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 746/1492] 10.222.0.65 () {58 vars in 1023 bytes} [Tue May 10 20:11:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:11:09.233599 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed610f2-271e-4134-b14b-61d6db24387e None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:11:09.244008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed610f2-271e-4134-b14b-61d6db24387e None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:11:09.247535 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defe2b8-4f47-488a-97d7-a93ed7631872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:11:09.248163 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defe2b8-4f47-488a-97d7-a93ed7631872 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:11:09.248448 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defe2b8-4f47-488a-97d7-a93ed7631872 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:11:09.248723 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defe2b8-4f47-488a-97d7-a93ed7631872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:11:09.257851 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6defe2b8-4f47-488a-97d7-a93ed7631872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:11:09.259305 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6defe2b8-4f47-488a-97d7-a93ed7631872 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:11:09.268469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 747/1493] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:11:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:12:45.620487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb0a5e-f046-42c3-9081-a4b7d3867fb9 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:12:45.621067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb0a5e-f046-42c3-9081-a4b7d3867fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:12:45.621067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb0a5e-f046-42c3-9081-a4b7d3867fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:12:45.638618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-f3fb0a5e-f046-42c3-9081-a4b7d3867fb9 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:12:45.640313 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f3fb0a5e-f046-42c3-9081-a4b7d3867fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:12:45.647969 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 747/1494] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:12:45.657436 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97292c-1436-4247-9d02-70fd9f78a75f None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:12:45.657938 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97292c-1436-4247-9d02-70fd9f78a75f None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:12:45.658059 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97292c-1436-4247-9d02-70fd9f78a75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:12:45.675153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-ff97292c-1436-4247-9d02-70fd9f78a75f None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:12:45.676345 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-ff97292c-1436-4247-9d02-70fd9f78a75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:12:45.683976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 748/1495] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:12:45.689151 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb0a5e-f046-42c3-9081-a4b7d3867fb9 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:12:45.698538 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb0a5e-f046-42c3-9081-a4b7d3867fb9 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:12:45.703640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c41b31-34ac-41f5-8e6f-5ee510ec99af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:12:45.704287 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c41b31-34ac-41f5-8e6f-5ee510ec99af admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:12:45.704576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c41b31-34ac-41f5-8e6f-5ee510ec99af admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:12:45.704856 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c41b31-34ac-41f5-8e6f-5ee510ec99af admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:12:45.705583 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c41b31-34ac-41f5-8e6f-5ee510ec99af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:12:45.705894 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c41b31-34ac-41f5-8e6f-5ee510ec99af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:12:45.706947 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: WARNING py.warnings [None req-f4c41b31-34ac-41f5-8e6f-5ee510ec99af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 10 20:12:45.706947 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: warnings.warn(msg) May 10 20:12:45.706947 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: May 10 20:12:45.707821 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c41b31-34ac-41f5-8e6f-5ee510ec99af admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:12:45.718970 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 748/1496] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:12:45.726787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97292c-1436-4247-9d02-70fd9f78a75f None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:12:45.735905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97292c-1436-4247-9d02-70fd9f78a75f None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:12:45.740457 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d439b1-8992-4245-b4cd-2101e79fd1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:12:45.741126 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d439b1-8992-4245-b4cd-2101e79fd1f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:12:45.741407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d439b1-8992-4245-b4cd-2101e79fd1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:12:45.741679 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d439b1-8992-4245-b4cd-2101e79fd1f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:12:45.742411 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d439b1-8992-4245-b4cd-2101e79fd1f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:12:45.744181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: WARNING py.warnings [None req-59d439b1-8992-4245-b4cd-2101e79fd1f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 20:12:45.744181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: warnings.warn(msg) May 10 20:12:45.744181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: May 10 20:12:45.745385 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d439b1-8992-4245-b4cd-2101e79fd1f5 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:12:45.861942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 749/1497] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:12:45 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:12:45.866263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c41b31-34ac-41f5-8e6f-5ee510ec99af admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:12:45.877031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c41b31-34ac-41f5-8e6f-5ee510ec99af admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:12:45.935690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e9411-c43c-4c23-9d76-bb80f98bf5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:12:45.936805 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e9411-c43c-4c23-9d76-bb80f98bf5af admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:12:45.937092 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e9411-c43c-4c23-9d76-bb80f98bf5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:12:45.937371 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e9411-c43c-4c23-9d76-bb80f98bf5af admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:12:45.938350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e9411-c43c-4c23-9d76-bb80f98bf5af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:12:45.939365 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: WARNING py.warnings [None req-ac5e9411-c43c-4c23-9d76-bb80f98bf5af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 20:12:45.939365 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: warnings.warn(msg) May 10 20:12:45.939365 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: May 10 20:12:45.940565 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e9411-c43c-4c23-9d76-bb80f98bf5af admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:12:46.037301 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 749/1498] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:12:45 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:12:46.041341 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d439b1-8992-4245-b4cd-2101e79fd1f5 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:12:46.052212 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d439b1-8992-4245-b4cd-2101e79fd1f5 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:12:46.056852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3f492-8d21-4d8f-9e04-b64ff09e5a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:12:46.057497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3f492-8d21-4d8f-9e04-b64ff09e5a60 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:12:46.057777 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3f492-8d21-4d8f-9e04-b64ff09e5a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:12:46.058048 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3f492-8d21-4d8f-9e04-b64ff09e5a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:12:46.058804 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3f492-8d21-4d8f-9e04-b64ff09e5a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:12:46.059807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: WARNING py.warnings [None req-bca3f492-8d21-4d8f-9e04-b64ff09e5a60 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 10 20:12:46.059807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: warnings.warn(msg) May 10 20:12:46.059807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: May 10 20:12:46.060677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3f492-8d21-4d8f-9e04-b64ff09e5a60 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:12:46.073871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 750/1499] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:12:46.077866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e9411-c43c-4c23-9d76-bb80f98bf5af admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:12:46.088134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e9411-c43c-4c23-9d76-bb80f98bf5af admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:12:46.092671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7ae88-67b3-4d4b-ac60-07fa2cf8f872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:12:46.093316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7ae88-67b3-4d4b-ac60-07fa2cf8f872 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:12:46.093601 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7ae88-67b3-4d4b-ac60-07fa2cf8f872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:12:46.093877 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7ae88-67b3-4d4b-ac60-07fa2cf8f872 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:12:46.094659 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7ae88-67b3-4d4b-ac60-07fa2cf8f872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:12:46.095794 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: WARNING py.warnings [None req-30f7ae88-67b3-4d4b-ac60-07fa2cf8f872 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 10 20:12:46.095794 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: warnings.warn(msg) May 10 20:12:46.095794 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: May 10 20:12:46.096678 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7ae88-67b3-4d4b-ac60-07fa2cf8f872 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:12:46.110122 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 750/1500] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:12:46.113957 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3f492-8d21-4d8f-9e04-b64ff09e5a60 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:12:46.123488 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3f492-8d21-4d8f-9e04-b64ff09e5a60 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:12:46.129032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8d7c3-2939-4560-912a-d362888bcffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:12:46.129897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8d7c3-2939-4560-912a-d362888bcffd admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:12:46.130215 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8d7c3-2939-4560-912a-d362888bcffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:12:46.130493 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8d7c3-2939-4560-912a-d362888bcffd admin admin] PATH_INFO: `/v3/projects/8aed6ea3c37542018aae05377d7f7390/users/f62f3df8a4f04940913a1e723253e29c/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:12:46.145080 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8d7c3-2939-4560-912a-d362888bcffd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aed6ea3c37542018aae05377d7f7390, user_id=f62f3df8a4f04940913a1e723253e29c, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:12:46.146144 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: WARNING py.warnings [None req-05a8d7c3-2939-4560-912a-d362888bcffd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 20:12:46.146144 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: warnings.warn(msg) May 10 20:12:46.146144 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: May 10 20:12:46.148052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8d7c3-2939-4560-912a-d362888bcffd admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:12:46.221190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 751/1501] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:12:46 2022] PUT /identity/v3/projects/8aed6ea3c37542018aae05377d7f7390/users/f62f3df8a4f04940913a1e723253e29c/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:12:46.232215 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e20f29-a4d2-4e12-ad44-60cfdfbc6416 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:12:46.232506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e20f29-a4d2-4e12-ad44-60cfdfbc6416 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:12:46.232784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e20f29-a4d2-4e12-ad44-60cfdfbc6416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:12:46.258960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-e4e20f29-a4d2-4e12-ad44-60cfdfbc6416 None None] MFA Rules not processed for user `f62f3df8a4f04940913a1e723253e29c`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:12:46.260152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-e4e20f29-a4d2-4e12-ad44-60cfdfbc6416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:12:46.320942 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 751/1502] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:12:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:12:46.334202 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6640e7-6823-407d-9407-b075b3eb56ce None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:12:46.334500 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6640e7-6823-407d-9407-b075b3eb56ce None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:12:46.334833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6640e7-6823-407d-9407-b075b3eb56ce None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:12:46.336244 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 752/1503] 10.222.0.65 () {54 vars in 963 bytes} [Tue May 10 20:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:12:46.341144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f815edd-b57f-4168-903a-b2f3c1a01a56 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:12:46.341437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f815edd-b57f-4168-903a-b2f3c1a01a56 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:12:46.341718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f815edd-b57f-4168-903a-b2f3c1a01a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:12:46.361215 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-1f815edd-b57f-4168-903a-b2f3c1a01a56 None None] MFA Rules not processed for user `0e366ec240e5464d8c5e92f807123c23`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:12:46.362496 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1f815edd-b57f-4168-903a-b2f3c1a01a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:12:46.410322 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 752/1504] 10.222.0.65 () {58 vars in 1060 bytes} [Tue May 10 20:12:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:12:46.417683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6640e7-6823-407d-9407-b075b3eb56ce None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:12:46.429133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6640e7-6823-407d-9407-b075b3eb56ce None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:12:46.434619 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39104389-b04f-4150-a9ee-09148f3c6e5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:12:46.435275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39104389-b04f-4150-a9ee-09148f3c6e5c service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:12:46.435558 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39104389-b04f-4150-a9ee-09148f3c6e5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:12:46.435830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39104389-b04f-4150-a9ee-09148f3c6e5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:12:46.447870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39104389-b04f-4150-a9ee-09148f3c6e5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:12:46.449288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39104389-b04f-4150-a9ee-09148f3c6e5c service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:12:46.457914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 753/1505] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:12:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:12:46.499753 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8795dd-e3af-40d7-985c-500e50191979 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:12:46.500045 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8795dd-e3af-40d7-985c-500e50191979 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:12:46.500324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8795dd-e3af-40d7-985c-500e50191979 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:12:46.501601 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 753/1506] 10.222.0.65 () {52 vars in 829 bytes} [Tue May 10 20:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:12:46.505169 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39104389-b04f-4150-a9ee-09148f3c6e5c service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:12:46.564973 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39104389-b04f-4150-a9ee-09148f3c6e5c service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:12:46.569468 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77cf0a-f650-4a3b-8139-96ad950f4003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:12:46.570350 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77cf0a-f650-4a3b-8139-96ad950f4003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:12:46.570637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77cf0a-f650-4a3b-8139-96ad950f4003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:12:46.570912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77cf0a-f650-4a3b-8139-96ad950f4003 admin admin] PATH_INFO: `/v3/users/f62f3df8a4f04940913a1e723253e29c` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:12:46.572101 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77cf0a-f650-4a3b-8139-96ad950f4003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f62f3df8a4f04940913a1e723253e29c)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:12:46.573112 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: WARNING py.warnings [None req-bd77cf0a-f650-4a3b-8139-96ad950f4003 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 20:12:46.573112 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: warnings.warn(msg) May 10 20:12:46.573112 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: May 10 20:12:46.574365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77cf0a-f650-4a3b-8139-96ad950f4003 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:12:46.747010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-bd77cf0a-f650-4a3b-8139-96ad950f4003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f62f3df8a4f04940913a1e723253e29c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:12:46.747696 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-bd77cf0a-f650-4a3b-8139-96ad950f4003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f62f3df8a4f04940913a1e723253e29c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:12:46.748433 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-bd77cf0a-f650-4a3b-8139-96ad950f4003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f62f3df8a4f04940913a1e723253e29c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:12:46.766388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-bd77cf0a-f650-4a3b-8139-96ad950f4003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f62f3df8a4f04940913a1e723253e29c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:12:46.778366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-bd77cf0a-f650-4a3b-8139-96ad950f4003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f62f3df8a4f04940913a1e723253e29c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:12:46.848481 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 754/1507] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:12:46 2022] DELETE /identity/v3/users/f62f3df8a4f04940913a1e723253e29c => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:12:46.853074 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0940d-49db-464e-809b-a1f7a405a88e None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:12:46.853813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0940d-49db-464e-809b-a1f7a405a88e None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:12:46.857415 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0940d-49db-464e-809b-a1f7a405a88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:12:46.871288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-2ff0940d-49db-464e-809b-a1f7a405a88e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:12:46.872472 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-2ff0940d-49db-464e-809b-a1f7a405a88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:12:46.929703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 754/1508] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:12:46.939554 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77cf0a-f650-4a3b-8139-96ad950f4003 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:12:46.940943 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-bd77cf0a-f650-4a3b-8139-96ad950f4003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:12:46.996385 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77cf0a-f650-4a3b-8139-96ad950f4003 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:12:47.000054 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d8066-bcae-4693-9c47-912eefd65517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:12:47.000701 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d8066-bcae-4693-9c47-912eefd65517 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:12:47.000991 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d8066-bcae-4693-9c47-912eefd65517 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:12:47.001273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d8066-bcae-4693-9c47-912eefd65517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:12:47.009987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d8066-bcae-4693-9c47-912eefd65517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:12:47.011456 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d8066-bcae-4693-9c47-912eefd65517 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:12:47.021140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 755/1509] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:12:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:12:47.491550 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff0940d-49db-464e-809b-a1f7a405a88e None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:12:47.492953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-2ff0940d-49db-464e-809b-a1f7a405a88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:12:47.505440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff0940d-49db-464e-809b-a1f7a405a88e None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:12:47.510416 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dadda4-81a3-4965-88df-d4cffa1b4308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:12:47.511269 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dadda4-81a3-4965-88df-d4cffa1b4308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:12:47.511552 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dadda4-81a3-4965-88df-d4cffa1b4308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:12:47.511830 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dadda4-81a3-4965-88df-d4cffa1b4308 admin admin] PATH_INFO: `/v3/projects/8aed6ea3c37542018aae05377d7f7390` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:12:47.513025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dadda4-81a3-4965-88df-d4cffa1b4308 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aed6ea3c37542018aae05377d7f7390)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:12:47.514048 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: WARNING py.warnings [None req-89dadda4-81a3-4965-88df-d4cffa1b4308 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 20:12:47.514048 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: warnings.warn(msg) May 10 20:12:47.514048 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: May 10 20:12:47.516261 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dadda4-81a3-4965-88df-d4cffa1b4308 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:12:47.637098 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-89dadda4-81a3-4965-88df-d4cffa1b4308 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aed6ea3c37542018aae05377d7f7390', 'request_id': 'req-89dadda4-81a3-4965-88df-d4cffa1b4308'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:12:47.786892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-89dadda4-81a3-4965-88df-d4cffa1b4308 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aed6ea3c37542018aae05377d7f7390', 'request_id': 'req-89dadda4-81a3-4965-88df-d4cffa1b4308'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:12:47.794136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-89dadda4-81a3-4965-88df-d4cffa1b4308 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aed6ea3c37542018aae05377d7f7390', 'request_id': 'req-89dadda4-81a3-4965-88df-d4cffa1b4308'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:12:47.814809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-89dadda4-81a3-4965-88df-d4cffa1b4308 admin admin] The token cache is being invalidate because project 8aed6ea3c37542018aae05377d7f7390 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:12:47.837503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-89dadda4-81a3-4965-88df-d4cffa1b4308 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:12:47.837503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-89dadda4-81a3-4965-88df-d4cffa1b4308 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:12:47.837503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 755/1510] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:12:47 2022] DELETE /identity/v3/projects/8aed6ea3c37542018aae05377d7f7390 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:14:31.344911 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1784972-e159-4b1f-afd8-e9f9fd8f598c None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:31.345513 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1784972-e159-4b1f-afd8-e9f9fd8f598c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:31.345513 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1784972-e159-4b1f-afd8-e9f9fd8f598c None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:31.346876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 20:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:14:31.351718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e8a38-fe29-4c2f-a618-523cffbb0131 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:31.352178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e8a38-fe29-4c2f-a618-523cffbb0131 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:31.352284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e8a38-fe29-4c2f-a618-523cffbb0131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:31.370469 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-038e8a38-fe29-4c2f-a618-523cffbb0131 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:14:31.371669 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-038e8a38-fe29-4c2f-a618-523cffbb0131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:14:31.428534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 20:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:14:31.595370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1784972-e159-4b1f-afd8-e9f9fd8f598c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:14:31.596965 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-f1784972-e159-4b1f-afd8-e9f9fd8f598c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:14:31.670215 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1784972-e159-4b1f-afd8-e9f9fd8f598c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:14:31.675576 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f453a-7012-435b-a020-950efbbd49fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:14:31.676225 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f453a-7012-435b-a020-950efbbd49fd service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:31.676510 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f453a-7012-435b-a020-950efbbd49fd service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:31.676786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f453a-7012-435b-a020-950efbbd49fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:31.685330 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f453a-7012-435b-a020-950efbbd49fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:14:31.686948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f453a-7012-435b-a020-950efbbd49fd service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:14:31.696518 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 757/1513] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:14:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:14:32.217619 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ce770-e5b5-4ef4-8ff9-2ee34874d90c None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:32.217943 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ce770-e5b5-4ef4-8ff9-2ee34874d90c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:32.218319 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ce770-e5b5-4ef4-8ff9-2ee34874d90c None None] PATH_INFO: `/v3` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:32.219755 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 757/1514] 10.222.0.83 () {54 vars in 925 bytes} [Tue May 10 20:14:32 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:14:32.225929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb21a5d-6d07-4fe3-8176-d3793e28d78d None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:32.226283 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb21a5d-6d07-4fe3-8176-d3793e28d78d None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:32.226572 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb21a5d-6d07-4fe3-8176-d3793e28d78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:32.247375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-bbb21a5d-6d07-4fe3-8176-d3793e28d78d None None] MFA Rules not processed for user `022093dea10d46478b3016395c712d93`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:14:32.248595 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-bbb21a5d-6d07-4fe3-8176-d3793e28d78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:14:32.311965 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 758/1515] 10.222.0.83 () {58 vars in 1015 bytes} [Tue May 10 20:14:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:14:32.617766 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94742b-5caf-4c16-bdec-e83e5cdffe2a None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:32.618339 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94742b-5caf-4c16-bdec-e83e5cdffe2a None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:32.618339 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94742b-5caf-4c16-bdec-e83e5cdffe2a None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:32.619716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 758/1516] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 20:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:14:32.624964 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60711180-5e64-4d1e-aa41-2846022bd152 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:32.625420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60711180-5e64-4d1e-aa41-2846022bd152 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:32.625554 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60711180-5e64-4d1e-aa41-2846022bd152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:32.643044 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-60711180-5e64-4d1e-aa41-2846022bd152 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:14:32.644261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-60711180-5e64-4d1e-aa41-2846022bd152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:14:32.652019 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 759/1517] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 20:14:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:14:32.802860 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a94742b-5caf-4c16-bdec-e83e5cdffe2a None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:14:32.817616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a94742b-5caf-4c16-bdec-e83e5cdffe2a None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:14:32.823043 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de59c3-3db4-4bf6-8f7e-695781ec17e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:14:32.823669 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de59c3-3db4-4bf6-8f7e-695781ec17e1 service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:32.823953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de59c3-3db4-4bf6-8f7e-695781ec17e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:32.824228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de59c3-3db4-4bf6-8f7e-695781ec17e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:32.832086 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de59c3-3db4-4bf6-8f7e-695781ec17e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:14:32.833668 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de59c3-3db4-4bf6-8f7e-695781ec17e1 service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:14:32.843302 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 759/1518] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:14:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:14:36.325216 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e80bac-17ff-42c3-8cd6-ad097754c151 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:36.325733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e80bac-17ff-42c3-8cd6-ad097754c151 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:36.325733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e80bac-17ff-42c3-8cd6-ad097754c151 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:36.327045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 760/1519] 10.222.0.65 () {54 vars in 960 bytes} [Tue May 10 20:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:14:36.332340 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5950b-fd05-4f75-b91d-1077539acb7e None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:36.332795 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5950b-fd05-4f75-b91d-1077539acb7e None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:36.332899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5950b-fd05-4f75-b91d-1077539acb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:36.351303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-3cb5950b-fd05-4f75-b91d-1077539acb7e None None] MFA Rules not processed for user `022093dea10d46478b3016395c712d93`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:14:36.352514 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-3cb5950b-fd05-4f75-b91d-1077539acb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:14:36.361125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 760/1520] 10.222.0.65 () {58 vars in 1057 bytes} [Tue May 10 20:14:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:14:36.370518 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e80bac-17ff-42c3-8cd6-ad097754c151 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:14:36.390526 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e80bac-17ff-42c3-8cd6-ad097754c151 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:14:36.395391 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569c031-d16f-4201-b5e9-361850c02e0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022093dea10d46478b3016395c712d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:14:36.396054 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569c031-d16f-4201-b5e9-361850c02e0d service placement] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:36.396336 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569c031-d16f-4201-b5e9-361850c02e0d service placement] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:36.396610 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569c031-d16f-4201-b5e9-361850c02e0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:36.405254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569c031-d16f-4201-b5e9-361850c02e0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:14:36.406859 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569c031-d16f-4201-b5e9-361850c02e0d service placement] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:14:36.416221 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 761/1521] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:14:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:14:44.149579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68354d-5064-4fb7-a624-662c4148bb4b None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:44.150228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68354d-5064-4fb7-a624-662c4148bb4b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:44.150228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68354d-5064-4fb7-a624-662c4148bb4b None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:44.151587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 20:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:14:44.156410 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d792a5-b0d1-4458-858e-5ea1c14ae70b None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:44.156872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d792a5-b0d1-4458-858e-5ea1c14ae70b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:44.157001 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d792a5-b0d1-4458-858e-5ea1c14ae70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:44.174567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-01d792a5-b0d1-4458-858e-5ea1c14ae70b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:14:44.175764 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-01d792a5-b0d1-4458-858e-5ea1c14ae70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:14:44.184312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 20:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:14:44.333914 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68354d-5064-4fb7-a624-662c4148bb4b None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:14:44.348024 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68354d-5064-4fb7-a624-662c4148bb4b None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:14:44.353414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46704dc-208a-47bc-b04f-9aa1d0eceb18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:14:44.354046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46704dc-208a-47bc-b04f-9aa1d0eceb18 service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:44.354354 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46704dc-208a-47bc-b04f-9aa1d0eceb18 service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:44.354630 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46704dc-208a-47bc-b04f-9aa1d0eceb18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:44.363557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46704dc-208a-47bc-b04f-9aa1d0eceb18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:14:44.365136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46704dc-208a-47bc-b04f-9aa1d0eceb18 service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:14:44.375051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 762/1524] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:14:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:14:45.281945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa198b4-26fd-4073-89cf-94f914af260b None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:45.282612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa198b4-26fd-4073-89cf-94f914af260b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:45.282612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa198b4-26fd-4073-89cf-94f914af260b None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:45.283899 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 20:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:14:45.288630 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8cbf5-4bf2-457c-bf92-529604417efc None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:45.289081 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8cbf5-4bf2-457c-bf92-529604417efc None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:45.289186 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8cbf5-4bf2-457c-bf92-529604417efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:45.307113 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-02b8cbf5-4bf2-457c-bf92-529604417efc None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:14:45.308292 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-02b8cbf5-4bf2-457c-bf92-529604417efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:14:45.315939 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 20:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:14:45.467701 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa198b4-26fd-4073-89cf-94f914af260b None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:14:45.482038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa198b4-26fd-4073-89cf-94f914af260b None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:14:45.487433 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e66e0d-6043-4fb1-a607-a73d2c86d912 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:14:45.488062 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e66e0d-6043-4fb1-a607-a73d2c86d912 service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:45.488345 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e66e0d-6043-4fb1-a607-a73d2c86d912 service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:45.488620 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e66e0d-6043-4fb1-a607-a73d2c86d912 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:45.496519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e66e0d-6043-4fb1-a607-a73d2c86d912 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:14:45.498110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e66e0d-6043-4fb1-a607-a73d2c86d912 service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:14:45.507195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 764/1527] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:14:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:14:49.718172 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7afc2-1f65-474d-95ad-a57f519f5d2b None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:49.718733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7afc2-1f65-474d-95ad-a57f519f5d2b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:49.718733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7afc2-1f65-474d-95ad-a57f519f5d2b None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:49.720103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 20:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:14:49.725445 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793388b0-64bb-4612-b649-651f466126b6 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:49.725904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793388b0-64bb-4612-b649-651f466126b6 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:49.726014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793388b0-64bb-4612-b649-651f466126b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:49.744658 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-793388b0-64bb-4612-b649-651f466126b6 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:14:49.745843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-793388b0-64bb-4612-b649-651f466126b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:14:49.753407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 20:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:14:49.788731 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7afc2-1f65-474d-95ad-a57f519f5d2b None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:14:49.791040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c6e7afc2-1f65-474d-95ad-a57f519f5d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:14:49.846380 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7afc2-1f65-474d-95ad-a57f519f5d2b None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:14:49.849877 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d2091-5fb3-4761-a26b-6dcedc380de0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:14:49.850547 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d2091-5fb3-4761-a26b-6dcedc380de0 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:49.850830 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d2091-5fb3-4761-a26b-6dcedc380de0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:49.851106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d2091-5fb3-4761-a26b-6dcedc380de0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:49.860202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d2091-5fb3-4761-a26b-6dcedc380de0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:14:49.861622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d2091-5fb3-4761-a26b-6dcedc380de0 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:14:49.871233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 765/1530] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:14:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:14:52.084564 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be950c9-e7d4-4de4-8406-4948fa0d232e None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:52.085116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be950c9-e7d4-4de4-8406-4948fa0d232e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:52.085229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be950c9-e7d4-4de4-8406-4948fa0d232e None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:52.086577 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 20:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:14:52.092303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f5873-8aa6-45ef-8330-23d7b6e08e32 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:52.092732 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f5873-8aa6-45ef-8330-23d7b6e08e32 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:52.092874 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f5873-8aa6-45ef-8330-23d7b6e08e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:52.112108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-853f5873-8aa6-45ef-8330-23d7b6e08e32 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:14:52.113316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-853f5873-8aa6-45ef-8330-23d7b6e08e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:14:52.121213 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 20:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:14:52.156360 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be950c9-e7d4-4de4-8406-4948fa0d232e None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:14:52.166952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be950c9-e7d4-4de4-8406-4948fa0d232e None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:14:52.170458 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114ee4d-da8c-41e4-90ef-20eb233ebd14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:14:52.171084 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114ee4d-da8c-41e4-90ef-20eb233ebd14 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:52.171370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114ee4d-da8c-41e4-90ef-20eb233ebd14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:52.171646 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114ee4d-da8c-41e4-90ef-20eb233ebd14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:52.180288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114ee4d-da8c-41e4-90ef-20eb233ebd14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:14:52.181969 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114ee4d-da8c-41e4-90ef-20eb233ebd14 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:14:52.191326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 767/1533] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:14:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:14:56.079068 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c6ee3-5d74-4587-abdb-f8a99f4c09c7 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:56.079622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c6ee3-5d74-4587-abdb-f8a99f4c09c7 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:56.079622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c6ee3-5d74-4587-abdb-f8a99f4c09c7 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:56.080975 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 20:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:14:56.086350 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77e235-6149-4246-aef4-fb4eb2927275 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:56.087420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77e235-6149-4246-aef4-fb4eb2927275 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:56.087532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77e235-6149-4246-aef4-fb4eb2927275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:56.104915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-8b77e235-6149-4246-aef4-fb4eb2927275 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:14:56.106114 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-8b77e235-6149-4246-aef4-fb4eb2927275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:14:56.113701 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 20:14:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:14:56.383200 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c6ee3-5d74-4587-abdb-f8a99f4c09c7 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:14:56.385475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-622c6ee3-5d74-4587-abdb-f8a99f4c09c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:14:56.441591 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c6ee3-5d74-4587-abdb-f8a99f4c09c7 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:14:56.445156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e119a-8c4d-4726-a2d2-ea1c663fcdba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:14:56.445805 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e119a-8c4d-4726-a2d2-ea1c663fcdba service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:56.446150 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e119a-8c4d-4726-a2d2-ea1c663fcdba service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:56.446958 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e119a-8c4d-4726-a2d2-ea1c663fcdba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:56.456558 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e119a-8c4d-4726-a2d2-ea1c663fcdba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:14:56.457983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e119a-8c4d-4726-a2d2-ea1c663fcdba service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:14:56.467180 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 768/1536] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:14:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:14:58.736258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebae723-d500-44b8-8281-64be9aaad44c None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:58.736772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebae723-d500-44b8-8281-64be9aaad44c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:58.736772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebae723-d500-44b8-8281-64be9aaad44c None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:58.738126 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 20:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:14:58.743305 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ad0f0-a450-448d-a3a1-a06870c0e91e None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:58.743732 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ad0f0-a450-448d-a3a1-a06870c0e91e None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:58.743860 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ad0f0-a450-448d-a3a1-a06870c0e91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:58.761735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-b43ad0f0-a450-448d-a3a1-a06870c0e91e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:14:58.762955 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b43ad0f0-a450-448d-a3a1-a06870c0e91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:14:58.770893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 20:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:14:59.040999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebae723-d500-44b8-8281-64be9aaad44c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:14:59.042668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4ebae723-d500-44b8-8281-64be9aaad44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:14:59.055412 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebae723-d500-44b8-8281-64be9aaad44c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:14:59.058823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc7ff5-fdd6-40b1-9a6d-634de29e7777 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:14:59.059441 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc7ff5-fdd6-40b1-9a6d-634de29e7777 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:14:59.059721 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc7ff5-fdd6-40b1-9a6d-634de29e7777 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:14:59.059995 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc7ff5-fdd6-40b1-9a6d-634de29e7777 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:14:59.068571 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc7ff5-fdd6-40b1-9a6d-634de29e7777 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:14:59.069962 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc7ff5-fdd6-40b1-9a6d-634de29e7777 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:14:59.079500 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 770/1539] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:14:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:30.676538 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2f58a-62d2-46de-878b-84eb49419052 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:30.677216 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2f58a-62d2-46de-878b-84eb49419052 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:30.677216 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2f58a-62d2-46de-878b-84eb49419052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:30.705267 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-26b2f58a-62d2-46de-878b-84eb49419052 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:30.706558 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-26b2f58a-62d2-46de-878b-84eb49419052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:30.722107 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 770/1540] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:30.733916 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1370d-4adf-4e8c-9f46-afd0b4d6b63b None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:30.734523 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1370d-4adf-4e8c-9f46-afd0b4d6b63b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:30.734523 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1370d-4adf-4e8c-9f46-afd0b4d6b63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:30.761407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-a1e1370d-4adf-4e8c-9f46-afd0b4d6b63b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:30.762791 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a1e1370d-4adf-4e8c-9f46-afd0b4d6b63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:30.774881 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 771/1541] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:30.779899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2f58a-62d2-46de-878b-84eb49419052 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:30.790621 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2f58a-62d2-46de-878b-84eb49419052 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:30.798694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b266d-a921-45f2-ab35-22c9e10f1883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:30.799772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b266d-a921-45f2-ab35-22c9e10f1883 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:30.800057 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b266d-a921-45f2-ab35-22c9e10f1883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:30.800335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b266d-a921-45f2-ab35-22c9e10f1883 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:30.801078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b266d-a921-45f2-ab35-22c9e10f1883 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:16:30.801388 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b266d-a921-45f2-ab35-22c9e10f1883 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:30.807133 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b266d-a921-45f2-ab35-22c9e10f1883 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:30.821338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 771/1542] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:16:30.829901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1370d-4adf-4e8c-9f46-afd0b4d6b63b None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:30.840355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1370d-4adf-4e8c-9f46-afd0b4d6b63b None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:30.855756 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447a4a4-2573-45d6-a3db-5669f27a7884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:30.856568 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447a4a4-2573-45d6-a3db-5669f27a7884 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:30.856857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447a4a4-2573-45d6-a3db-5669f27a7884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:30.857137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447a4a4-2573-45d6-a3db-5669f27a7884 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:30.857862 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447a4a4-2573-45d6-a3db-5669f27a7884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:30.860019 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447a4a4-2573-45d6-a3db-5669f27a7884 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:30.980776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 772/1543] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:16:30 2022] POST /identity/v3/projects => generated 367 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:30.984956 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b266d-a921-45f2-ab35-22c9e10f1883 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:30.998381 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b266d-a921-45f2-ab35-22c9e10f1883 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:31.060533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38d7f7-cd03-43c3-993d-3aa1fe743622 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.060898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38d7f7-cd03-43c3-993d-3aa1fe743622 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.061117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38d7f7-cd03-43c3-993d-3aa1fe743622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.076192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a1e70-74b1-43bc-a99f-10a246a00cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:31.077080 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a1e70-74b1-43bc-a99f-10a246a00cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.077373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a1e70-74b1-43bc-a99f-10a246a00cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.077654 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a1e70-74b1-43bc-a99f-10a246a00cc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.078413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a1e70-74b1-43bc-a99f-10a246a00cc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:31.080381 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a1e70-74b1-43bc-a99f-10a246a00cc6 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:31.097674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-3e38d7f7-cd03-43c3-993d-3aa1fe743622 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:31.100259 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3e38d7f7-cd03-43c3-993d-3aa1fe743622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:31.119069 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 773/1544] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:16:31.128920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe98d1-5801-49ef-af93-cae8a053713a None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.134069 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe98d1-5801-49ef-af93-cae8a053713a None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.134069 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe98d1-5801-49ef-af93-cae8a053713a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.157097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-99fe98d1-5801-49ef-af93-cae8a053713a None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:31.158343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-99fe98d1-5801-49ef-af93-cae8a053713a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:31.164961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 772/1545] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:16:30 2022] POST /identity/v3/users => generated 284 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:31.169178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a1e70-74b1-43bc-a99f-10a246a00cc6 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:31.177297 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 774/1546] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:16:31.182233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a1e70-74b1-43bc-a99f-10a246a00cc6 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:31.182395 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe98d1-5801-49ef-af93-cae8a053713a None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:31.191758 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe98d1-5801-49ef-af93-cae8a053713a None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:31.196924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634daf82-6adb-4af0-a892-6b660e52fb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:31.197630 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634daf82-6adb-4af0-a892-6b660e52fb31 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.198041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634daf82-6adb-4af0-a892-6b660e52fb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.198346 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634daf82-6adb-4af0-a892-6b660e52fb31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.199690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634daf82-6adb-4af0-a892-6b660e52fb31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:31.201347 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634daf82-6adb-4af0-a892-6b660e52fb31 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:31.204859 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d811f4c-403b-4584-bf74-4afac712aea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:31.205556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d811f4c-403b-4584-bf74-4afac712aea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.205840 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d811f4c-403b-4584-bf74-4afac712aea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.206137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d811f4c-403b-4584-bf74-4afac712aea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.206901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d811f4c-403b-4584-bf74-4afac712aea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:16:31.207210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d811f4c-403b-4584-bf74-4afac712aea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:31.209062 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: WARNING py.warnings [None req-2d811f4c-403b-4584-bf74-4afac712aea5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 10 20:16:31.209062 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: warnings.warn(msg) May 10 20:16:31.209062 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: May 10 20:16:31.209989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d811f4c-403b-4584-bf74-4afac712aea5 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:31.220053 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 773/1547] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:31.223868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634daf82-6adb-4af0-a892-6b660e52fb31 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:31.234739 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 775/1548] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 20:16:31.235667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d811f4c-403b-4584-bf74-4afac712aea5 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:31.240722 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634daf82-6adb-4af0-a892-6b660e52fb31 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:31.250582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d811f4c-403b-4584-bf74-4afac712aea5 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:31.254505 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4ddad-0ef9-407e-b272-25470e1b27a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:31.255994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4ddad-0ef9-407e-b272-25470e1b27a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.256398 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4ddad-0ef9-407e-b272-25470e1b27a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.256787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4ddad-0ef9-407e-b272-25470e1b27a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.257843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4ddad-0ef9-407e-b272-25470e1b27a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:31.259513 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4ddad-0ef9-407e-b272-25470e1b27a9 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:31.260055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52782071-6728-45b0-a987-ada85e9aa823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:31.260738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52782071-6728-45b0-a987-ada85e9aa823 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.261024 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52782071-6728-45b0-a987-ada85e9aa823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.261304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52782071-6728-45b0-a987-ada85e9aa823 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.262021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52782071-6728-45b0-a987-ada85e9aa823 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:31.264018 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52782071-6728-45b0-a987-ada85e9aa823 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:31.279957 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 774/1549] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:16:31.284679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4ddad-0ef9-407e-b272-25470e1b27a9 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:31.300500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4ddad-0ef9-407e-b272-25470e1b27a9 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:31.315892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19544321-d01d-4275-82f8-c712a7bda168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:31.316825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19544321-d01d-4275-82f8-c712a7bda168 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.317114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19544321-d01d-4275-82f8-c712a7bda168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.317393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19544321-d01d-4275-82f8-c712a7bda168 admin admin] PATH_INFO: `/v3/projects/051ca1834f2a4c2a84e885a473dab12b/users/503075ec6bb947a6b21d009de442cec7/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.340881 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19544321-d01d-4275-82f8-c712a7bda168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=051ca1834f2a4c2a84e885a473dab12b, user_id=503075ec6bb947a6b21d009de442cec7, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:31.341975 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: WARNING py.warnings [None req-19544321-d01d-4275-82f8-c712a7bda168 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 20:16:31.341975 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: warnings.warn(msg) May 10 20:16:31.341975 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: May 10 20:16:31.344287 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19544321-d01d-4275-82f8-c712a7bda168 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:31.372554 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 776/1550] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:16:31 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:31.374056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdacfd5-8998-4dad-8257-45d671bef914 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.374387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdacfd5-8998-4dad-8257-45d671bef914 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.374651 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdacfd5-8998-4dad-8257-45d671bef914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.402643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-6cdacfd5-8998-4dad-8257-45d671bef914 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:31.403844 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-6cdacfd5-8998-4dad-8257-45d671bef914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:31.422682 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 775/1551] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:31 2022] PUT /identity/v3/projects/051ca1834f2a4c2a84e885a473dab12b/users/503075ec6bb947a6b21d009de442cec7/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:31.423523 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19544321-d01d-4275-82f8-c712a7bda168 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:31.509259 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 777/1552] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:31.510983 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3f403-150e-4e6c-a7c8-ee2fbb7458ca None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.511505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3f403-150e-4e6c-a7c8-ee2fbb7458ca None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.512132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3f403-150e-4e6c-a7c8-ee2fbb7458ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.528035 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19544321-d01d-4275-82f8-c712a7bda168 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:31.536465 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d081d9-b10c-4265-8fed-7f8590b7954b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:31.537473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d081d9-b10c-4265-8fed-7f8590b7954b admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.537780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d081d9-b10c-4265-8fed-7f8590b7954b admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.538064 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d081d9-b10c-4265-8fed-7f8590b7954b admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.538809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d081d9-b10c-4265-8fed-7f8590b7954b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:31.540847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d081d9-b10c-4265-8fed-7f8590b7954b admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:31.553066 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-9ec3f403-150e-4e6c-a7c8-ee2fbb7458ca None None] MFA Rules not processed for user `503075ec6bb947a6b21d009de442cec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:31.554323 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-9ec3f403-150e-4e6c-a7c8-ee2fbb7458ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:31.623873 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 778/1553] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:31.623873 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6afa94-2643-4fc6-9a15-b7f73a3f67e2 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.623873 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6afa94-2643-4fc6-9a15-b7f73a3f67e2 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.623873 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6afa94-2643-4fc6-9a15-b7f73a3f67e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.643916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 776/1554] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:16:31 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:31.643916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c6a4e-3fdf-437a-871e-62cf17c7fa0f None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.643916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c6a4e-3fdf-437a-871e-62cf17c7fa0f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.643916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c6a4e-3fdf-437a-871e-62cf17c7fa0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.645464 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-0d6afa94-2643-4fc6-9a15-b7f73a3f67e2 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:31.646726 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-0d6afa94-2643-4fc6-9a15-b7f73a3f67e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:31.655370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 779/1555] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:31.656637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc505a-3aa9-4f41-85ff-c8ecb66b4e39 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.656918 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc505a-3aa9-4f41-85ff-c8ecb66b4e39 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.657198 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc505a-3aa9-4f41-85ff-c8ecb66b4e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.663556 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-6c6c6a4e-3fdf-437a-871e-62cf17c7fa0f None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:31.664778 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-6c6c6a4e-3fdf-437a-871e-62cf17c7fa0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:31.677534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 777/1556] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:31.677534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08bcf2-b1a4-4608-b357-e81938505eef None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.677534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08bcf2-b1a4-4608-b357-e81938505eef None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.677534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08bcf2-b1a4-4608-b357-e81938505eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.680113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-b2dc505a-3aa9-4f41-85ff-c8ecb66b4e39 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:31.681318 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b2dc505a-3aa9-4f41-85ff-c8ecb66b4e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:31.689275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 780/1557] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:31.690539 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fd660-4d3d-4c67-a61b-7d69c8b1c84b None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.690831 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fd660-4d3d-4c67-a61b-7d69c8b1c84b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.691106 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fd660-4d3d-4c67-a61b-7d69c8b1c84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.702458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-bd08bcf2-b1a4-4608-b357-e81938505eef None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:31.703505 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-bd08bcf2-b1a4-4608-b357-e81938505eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:31.708358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-219fd660-4d3d-4c67-a61b-7d69c8b1c84b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:31.709518 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-219fd660-4d3d-4c67-a61b-7d69c8b1c84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:31.711389 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 778/1558] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:31.712091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08bcf2-b1a4-4608-b357-e81938505eef None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:31.722469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 781/1559] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:31.723499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fd660-4d3d-4c67-a61b-7d69c8b1c84b None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:31.733573 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fd660-4d3d-4c67-a61b-7d69c8b1c84b None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:31.738262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541537ad-0a58-4624-99af-af0c5391bf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:31.738908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541537ad-0a58-4624-99af-af0c5391bf28 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.739186 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541537ad-0a58-4624-99af-af0c5391bf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.740087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541537ad-0a58-4624-99af-af0c5391bf28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.740817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541537ad-0a58-4624-99af-af0c5391bf28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:31.742451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541537ad-0a58-4624-99af-af0c5391bf28 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:31.752210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 782/1560] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:31.753433 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5646ce-911d-4c99-9902-5737d39e3ea7 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.753726 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5646ce-911d-4c99-9902-5737d39e3ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.754007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5646ce-911d-4c99-9902-5737d39e3ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.770752 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08bcf2-b1a4-4608-b357-e81938505eef None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:31.771020 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-bb5646ce-911d-4c99-9902-5737d39e3ea7 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:31.772241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-bb5646ce-911d-4c99-9902-5737d39e3ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:31.774447 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad72952-2b4d-4c77-9501-dfeda733f414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:31.775095 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad72952-2b4d-4c77-9501-dfeda733f414 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.775379 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad72952-2b4d-4c77-9501-dfeda733f414 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.775655 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad72952-2b4d-4c77-9501-dfeda733f414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.780117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 783/1561] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:31.782264 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f2214-0df9-4301-90e5-2321be7737a5 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.782264 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f2214-0df9-4301-90e5-2321be7737a5 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.782264 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f2214-0df9-4301-90e5-2321be7737a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.790432 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad72952-2b4d-4c77-9501-dfeda733f414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:31.792007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad72952-2b4d-4c77-9501-dfeda733f414 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:31.800219 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-2d3f2214-0df9-4301-90e5-2321be7737a5 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:31.801253 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 779/1562] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:31 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:31.812285 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-2d3f2214-0df9-4301-90e5-2321be7737a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:31.812856 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af37cf3-bfee-45f3-bcb5-cdc0291b801d None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.812856 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af37cf3-bfee-45f3-bcb5-cdc0291b801d None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.812856 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af37cf3-bfee-45f3-bcb5-cdc0291b801d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.813123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 784/1563] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:31.813123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f2214-0df9-4301-90e5-2321be7737a5 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:31.819936 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f2214-0df9-4301-90e5-2321be7737a5 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:31.824503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db9147-0edf-41b3-a849-a855341e871d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:31.825193 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db9147-0edf-41b3-a849-a855341e871d admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.825422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db9147-0edf-41b3-a849-a855341e871d admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.825696 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db9147-0edf-41b3-a849-a855341e871d admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.826458 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db9147-0edf-41b3-a849-a855341e871d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:16:31.826769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db9147-0edf-41b3-a849-a855341e871d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:31.828362 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db9147-0edf-41b3-a849-a855341e871d admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:31.839830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 785/1564] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:16:31.841046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f7adc-3086-4bee-bbc3-283117f975e4 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.841336 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f7adc-3086-4bee-bbc3-283117f975e4 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.841618 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f7adc-3086-4bee-bbc3-283117f975e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.894167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-1af37cf3-bfee-45f3-bcb5-cdc0291b801d None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:31.895257 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-cf9f7adc-3086-4bee-bbc3-283117f975e4 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:31.895396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1af37cf3-bfee-45f3-bcb5-cdc0291b801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:31.896769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-cf9f7adc-3086-4bee-bbc3-283117f975e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:31.903974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 780/1565] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:31.908477 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 786/1566] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:31.908615 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864eca8-adf2-4135-96f0-b72f04aa91e0 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.908713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f7adc-3086-4bee-bbc3-283117f975e4 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:31.908816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864eca8-adf2-4135-96f0-b72f04aa91e0 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.908816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864eca8-adf2-4135-96f0-b72f04aa91e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.926131 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f7adc-3086-4bee-bbc3-283117f975e4 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:31.928899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-4864eca8-adf2-4135-96f0-b72f04aa91e0 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:31.930105 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-4864eca8-adf2-4135-96f0-b72f04aa91e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:31.931688 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b406368-8711-45d4-a8ee-49238e05ec97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:31.932437 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b406368-8711-45d4-a8ee-49238e05ec97 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.932796 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b406368-8711-45d4-a8ee-49238e05ec97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.933151 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b406368-8711-45d4-a8ee-49238e05ec97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.933967 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b406368-8711-45d4-a8ee-49238e05ec97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:31.935697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b406368-8711-45d4-a8ee-49238e05ec97 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:31.938803 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 781/1567] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:31.943914 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a12302-d24e-429e-a44e-33ed15404e30 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.944262 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a12302-d24e-429e-a44e-33ed15404e30 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.944743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a12302-d24e-429e-a44e-33ed15404e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.949446 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 787/1568] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:31.949446 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b406368-8711-45d4-a8ee-49238e05ec97 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:31.959451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b406368-8711-45d4-a8ee-49238e05ec97 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:31.964278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9dd34-074a-446f-b8b2-a4ce684e71e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:31.965006 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9dd34-074a-446f-b8b2-a4ce684e71e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.965353 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9dd34-074a-446f-b8b2-a4ce684e71e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.965809 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9dd34-074a-446f-b8b2-a4ce684e71e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.966678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9dd34-074a-446f-b8b2-a4ce684e71e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:16:31.967050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9dd34-074a-446f-b8b2-a4ce684e71e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:31.967915 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-65a12302-d24e-429e-a44e-33ed15404e30 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:31.968898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9dd34-074a-446f-b8b2-a4ce684e71e2 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:31.969259 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-65a12302-d24e-429e-a44e-33ed15404e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:31.981337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 782/1569] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:31.981337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a12302-d24e-429e-a44e-33ed15404e30 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:31.986533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 788/1570] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:16:31.990128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a12302-d24e-429e-a44e-33ed15404e30 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:31.991233 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9dd34-074a-446f-b8b2-a4ce684e71e2 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:31.995385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50d045-c667-45bd-881a-5ed3aa26e6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:31.996128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50d045-c667-45bd-881a-5ed3aa26e6bb admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:31.996481 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50d045-c667-45bd-881a-5ed3aa26e6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:31.996827 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50d045-c667-45bd-881a-5ed3aa26e6bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:31.997702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50d045-c667-45bd-881a-5ed3aa26e6bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:31.998785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: WARNING py.warnings [None req-eb50d045-c667-45bd-881a-5ed3aa26e6bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 20:16:31.998785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: warnings.warn(msg) May 10 20:16:31.998785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: May 10 20:16:31.999989 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50d045-c667-45bd-881a-5ed3aa26e6bb admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:32.003988 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9dd34-074a-446f-b8b2-a4ce684e71e2 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:32.013658 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336653a7-e123-41e4-b8eb-bbffc4ee066b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:32.014413 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336653a7-e123-41e4-b8eb-bbffc4ee066b admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:32.014677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336653a7-e123-41e4-b8eb-bbffc4ee066b admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:32.014956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336653a7-e123-41e4-b8eb-bbffc4ee066b admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:32.015692 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336653a7-e123-41e4-b8eb-bbffc4ee066b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:16:32.016005 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336653a7-e123-41e4-b8eb-bbffc4ee066b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:32.017604 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336653a7-e123-41e4-b8eb-bbffc4ee066b admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:32.032012 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 789/1571] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:16:32.033010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336653a7-e123-41e4-b8eb-bbffc4ee066b admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:32.050033 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336653a7-e123-41e4-b8eb-bbffc4ee066b admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:32.055708 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85860ec8-756c-4bd8-9e3f-b7bced6b76b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:32.098052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85860ec8-756c-4bd8-9e3f-b7bced6b76b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:32.098052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85860ec8-756c-4bd8-9e3f-b7bced6b76b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:32.098052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85860ec8-756c-4bd8-9e3f-b7bced6b76b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:32.098052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85860ec8-756c-4bd8-9e3f-b7bced6b76b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:16:32.098052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85860ec8-756c-4bd8-9e3f-b7bced6b76b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:32.098052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85860ec8-756c-4bd8-9e3f-b7bced6b76b3 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:32.098735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 790/1572] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:16:32.098735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85860ec8-756c-4bd8-9e3f-b7bced6b76b3 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:32.098735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85860ec8-756c-4bd8-9e3f-b7bced6b76b3 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:32.098735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f0852-3a42-4a30-b7a1-1c34f95a4df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:32.099621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f0852-3a42-4a30-b7a1-1c34f95a4df5 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:32.099621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f0852-3a42-4a30-b7a1-1c34f95a4df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:32.099621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f0852-3a42-4a30-b7a1-1c34f95a4df5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:32.099621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f0852-3a42-4a30-b7a1-1c34f95a4df5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:16:32.099621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f0852-3a42-4a30-b7a1-1c34f95a4df5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:32.099621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f0852-3a42-4a30-b7a1-1c34f95a4df5 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:32.106855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 791/1573] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:16:32.107575 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f0852-3a42-4a30-b7a1-1c34f95a4df5 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:32.117760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f0852-3a42-4a30-b7a1-1c34f95a4df5 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:32.122491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0757e03-6edb-4143-ad15-14a11b3be7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:32.123514 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0757e03-6edb-4143-ad15-14a11b3be7f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:32.123719 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0757e03-6edb-4143-ad15-14a11b3be7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:32.124027 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0757e03-6edb-4143-ad15-14a11b3be7f1 admin admin] PATH_INFO: `/v3/projects/62ae41513d9d4505a06c4710f82d48ba/users/fd594cb249d6432a8ff3a768e74f982e/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:32.139827 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0757e03-6edb-4143-ad15-14a11b3be7f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62ae41513d9d4505a06c4710f82d48ba, user_id=fd594cb249d6432a8ff3a768e74f982e, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:32.142773 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0757e03-6edb-4143-ad15-14a11b3be7f1 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:32.144189 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 783/1574] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:16:31 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:32.145041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50d045-c667-45bd-881a-5ed3aa26e6bb admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:32.205776 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50d045-c667-45bd-881a-5ed3aa26e6bb admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:32.211054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c35c0b-d772-4d3e-b2df-7d66e151ebef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:32.211716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c35c0b-d772-4d3e-b2df-7d66e151ebef admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:32.212000 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c35c0b-d772-4d3e-b2df-7d66e151ebef admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:32.212278 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c35c0b-d772-4d3e-b2df-7d66e151ebef admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:32.213012 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c35c0b-d772-4d3e-b2df-7d66e151ebef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:16:32.213324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c35c0b-d772-4d3e-b2df-7d66e151ebef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:32.214954 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c35c0b-d772-4d3e-b2df-7d66e151ebef admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:32.226787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 784/1575] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:16:32.227555 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c35c0b-d772-4d3e-b2df-7d66e151ebef admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:32.236869 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c35c0b-d772-4d3e-b2df-7d66e151ebef admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:32.241463 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806f5c3-b71f-4124-9746-6b644e57228c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:32.242180 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806f5c3-b71f-4124-9746-6b644e57228c admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:32.242474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806f5c3-b71f-4124-9746-6b644e57228c admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:32.242754 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806f5c3-b71f-4124-9746-6b644e57228c admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:32.243434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806f5c3-b71f-4124-9746-6b644e57228c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:32.244625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 792/1576] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:32 2022] PUT /identity/v3/projects/62ae41513d9d4505a06c4710f82d48ba/users/fd594cb249d6432a8ff3a768e74f982e/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:32.245310 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806f5c3-b71f-4124-9746-6b644e57228c admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:32.245521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0757e03-6edb-4143-ad15-14a11b3be7f1 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:32.311159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0757e03-6edb-4143-ad15-14a11b3be7f1 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:32.315609 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316e6e8-9ffb-4986-988b-77041d63a830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:32.316262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316e6e8-9ffb-4986-988b-77041d63a830 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:32.316549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316e6e8-9ffb-4986-988b-77041d63a830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:32.316858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316e6e8-9ffb-4986-988b-77041d63a830 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:32.317572 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316e6e8-9ffb-4986-988b-77041d63a830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:32.319520 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316e6e8-9ffb-4986-988b-77041d63a830 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:32.337523 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 785/1577] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:16:32 2022] POST /identity/v3/projects => generated 393 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:32.338640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806f5c3-b71f-4124-9746-6b644e57228c admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:32.398624 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806f5c3-b71f-4124-9746-6b644e57228c admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:32.403452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9fc7fb-e84c-4797-9e9d-338873772d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:32.404112 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fc7fb-e84c-4797-9e9d-338873772d54 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:32.404395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fc7fb-e84c-4797-9e9d-338873772d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:32.404669 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fc7fb-e84c-4797-9e9d-338873772d54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:32.405388 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9fc7fb-e84c-4797-9e9d-338873772d54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:32.405819 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 793/1578] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:16:32 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:32.407463 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9fc7fb-e84c-4797-9e9d-338873772d54 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:32.409843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316e6e8-9ffb-4986-988b-77041d63a830 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:32.472289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316e6e8-9ffb-4986-988b-77041d63a830 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:32.476933 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731941ca-a128-461f-8b0d-d91739e132e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:32.477663 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731941ca-a128-461f-8b0d-d91739e132e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:32.477942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731941ca-a128-461f-8b0d-d91739e132e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:32.478250 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731941ca-a128-461f-8b0d-d91739e132e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:32.478945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731941ca-a128-461f-8b0d-d91739e132e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:32.480879 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731941ca-a128-461f-8b0d-d91739e132e9 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:32.500997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 786/1579] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:16:32 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:32.502321 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9fc7fb-e84c-4797-9e9d-338873772d54 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:32.563085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9fc7fb-e84c-4797-9e9d-338873772d54 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:32.566129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 794/1580] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:16:32 2022] POST /identity/v3/projects => generated 403 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:32.566943 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731941ca-a128-461f-8b0d-d91739e132e9 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:32.571174 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad6b64-b8b2-47b4-b7b5-aa4437a0f923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:32.571846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad6b64-b8b2-47b4-b7b5-aa4437a0f923 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:32.572132 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad6b64-b8b2-47b4-b7b5-aa4437a0f923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:32.572407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad6b64-b8b2-47b4-b7b5-aa4437a0f923 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:32.573103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad6b64-b8b2-47b4-b7b5-aa4437a0f923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:32.575074 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad6b64-b8b2-47b4-b7b5-aa4437a0f923 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:32.627918 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731941ca-a128-461f-8b0d-d91739e132e9 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:32.633013 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6652b-c407-4f81-b9b7-45f9443036b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:32.633658 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6652b-c407-4f81-b9b7-45f9443036b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:32.633936 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6652b-c407-4f81-b9b7-45f9443036b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:32.634242 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6652b-c407-4f81-b9b7-45f9443036b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:32.634932 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6652b-c407-4f81-b9b7-45f9443036b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:32.636812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6652b-c407-4f81-b9b7-45f9443036b4 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:32.673542 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 787/1581] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:16:32 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:32.675202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add7242-b5f3-4e14-bdd9-982d2d256ed1 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:32.675490 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add7242-b5f3-4e14-bdd9-982d2d256ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:32.675764 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add7242-b5f3-4e14-bdd9-982d2d256ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:32.703268 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-1add7242-b5f3-4e14-bdd9-982d2d256ed1 None None] MFA Rules not processed for user `fd594cb249d6432a8ff3a768e74f982e`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:32.704490 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1add7242-b5f3-4e14-bdd9-982d2d256ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:32.763294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 795/1582] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:16:32 2022] POST /identity/v3/projects => generated 357 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:32.764370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6652b-c407-4f81-b9b7-45f9443036b4 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:32.773754 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 788/1583] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:32.774857 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add7242-b5f3-4e14-bdd9-982d2d256ed1 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:32.787326 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add7242-b5f3-4e14-bdd9-982d2d256ed1 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:32.841271 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6652b-c407-4f81-b9b7-45f9443036b4 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:32.846384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44fe7f4-4716-492c-8fba-f2881f841253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:32.847197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fe7f4-4716-492c-8fba-f2881f841253 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:32.847616 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fe7f4-4716-492c-8fba-f2881f841253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:32.847956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fe7f4-4716-492c-8fba-f2881f841253 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:32.848898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44fe7f4-4716-492c-8fba-f2881f841253 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:32.849914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: WARNING py.warnings [None req-a44fe7f4-4716-492c-8fba-f2881f841253 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 20:16:32.849914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: warnings.warn(msg) May 10 20:16:32.849914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: May 10 20:16:32.851160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44fe7f4-4716-492c-8fba-f2881f841253 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:32.858091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3f133-e2f3-4dab-b6bc-b08f7010be21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:32.858792 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3f133-e2f3-4dab-b6bc-b08f7010be21 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:32.859080 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3f133-e2f3-4dab-b6bc-b08f7010be21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:32.859360 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3f133-e2f3-4dab-b6bc-b08f7010be21 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:32.860224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3f133-e2f3-4dab-b6bc-b08f7010be21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:32.862198 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3f133-e2f3-4dab-b6bc-b08f7010be21 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:32.918933 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 796/1584] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:16:32 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:32.919892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44fe7f4-4716-492c-8fba-f2881f841253 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:32.930414 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44fe7f4-4716-492c-8fba-f2881f841253 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:32.937098 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc0f84-c886-475f-93a2-fbe5ff7a5063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:32.938103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc0f84-c886-475f-93a2-fbe5ff7a5063 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:32.938545 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc0f84-c886-475f-93a2-fbe5ff7a5063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:32.938959 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc0f84-c886-475f-93a2-fbe5ff7a5063 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:32.940050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc0f84-c886-475f-93a2-fbe5ff7a5063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:32.943780 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc0f84-c886-475f-93a2-fbe5ff7a5063 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:32.948644 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 789/1585] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:16:32 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:32.952303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3f133-e2f3-4dab-b6bc-b08f7010be21 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:32.961032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3f133-e2f3-4dab-b6bc-b08f7010be21 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:32.966774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397a14b-6ec3-4e5c-8fc3-0034e1afd3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:32.969557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397a14b-6ec3-4e5c-8fc3-0034e1afd3a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:32.969557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397a14b-6ec3-4e5c-8fc3-0034e1afd3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:32.969557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397a14b-6ec3-4e5c-8fc3-0034e1afd3a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:32.969557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397a14b-6ec3-4e5c-8fc3-0034e1afd3a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:32.971779 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397a14b-6ec3-4e5c-8fc3-0034e1afd3a9 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.113257 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 797/1586] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:16:32 2022] POST /identity/v3/users => generated 294 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:33.114053 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc0f84-c886-475f-93a2-fbe5ff7a5063 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.124261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc0f84-c886-475f-93a2-fbe5ff7a5063 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.128840 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ceaec-ed59-4abc-be1d-462359afc399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.129488 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ceaec-ed59-4abc-be1d-462359afc399 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.129769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ceaec-ed59-4abc-be1d-462359afc399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.130700 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ceaec-ed59-4abc-be1d-462359afc399 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.131436 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ceaec-ed59-4abc-be1d-462359afc399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.133015 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ceaec-ed59-4abc-be1d-462359afc399 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.142002 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 790/1587] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:16:32 2022] POST /identity/v3/users => generated 302 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:33.142910 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397a14b-6ec3-4e5c-8fc3-0034e1afd3a9 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.146624 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 798/1588] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:33.146624 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ceaec-ed59-4abc-be1d-462359afc399 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.155460 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ceaec-ed59-4abc-be1d-462359afc399 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.155631 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397a14b-6ec3-4e5c-8fc3-0034e1afd3a9 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.160096 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c641fb-aef0-4242-853d-944cb74fee09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.160367 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9a405-9272-4010-9648-8c3e855e13b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.160741 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c641fb-aef0-4242-853d-944cb74fee09 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.161023 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c641fb-aef0-4242-853d-944cb74fee09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.161681 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9a405-9272-4010-9648-8c3e855e13b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.163904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c641fb-aef0-4242-853d-944cb74fee09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.168449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9a405-9272-4010-9648-8c3e855e13b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.168449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9a405-9272-4010-9648-8c3e855e13b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.168686 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c641fb-aef0-4242-853d-944cb74fee09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.168799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9a405-9272-4010-9648-8c3e855e13b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.168885 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c641fb-aef0-4242-853d-944cb74fee09 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.168960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9a405-9272-4010-9648-8c3e855e13b2 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.287943 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 791/1589] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:16:33 2022] POST /identity/v3/users => generated 279 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:33.289526 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9a405-9272-4010-9648-8c3e855e13b2 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.300799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9a405-9272-4010-9648-8c3e855e13b2 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.304496 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 799/1590] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:16:33 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:33.304496 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c641fb-aef0-4242-853d-944cb74fee09 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.361887 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cdba8-c018-47fd-9f3a-861fa985afd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.362603 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cdba8-c018-47fd-9f3a-861fa985afd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.362892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cdba8-c018-47fd-9f3a-861fa985afd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.363173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cdba8-c018-47fd-9f3a-861fa985afd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.364866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10cdba8-c018-47fd-9f3a-861fa985afd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.367032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10cdba8-c018-47fd-9f3a-861fa985afd9 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.368728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c641fb-aef0-4242-853d-944cb74fee09 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.374026 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c28d6-997e-4d53-9df2-b7ffee4d8f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.374705 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c28d6-997e-4d53-9df2-b7ffee4d8f0a admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.374983 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c28d6-997e-4d53-9df2-b7ffee4d8f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.375254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c28d6-997e-4d53-9df2-b7ffee4d8f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.376012 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c28d6-997e-4d53-9df2-b7ffee4d8f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.377596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c28d6-997e-4d53-9df2-b7ffee4d8f0a admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.377715 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 792/1591] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:33.378558 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cdba8-c018-47fd-9f3a-861fa985afd9 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.388231 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cdba8-c018-47fd-9f3a-861fa985afd9 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.391936 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 800/1592] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:33.391936 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c28d6-997e-4d53-9df2-b7ffee4d8f0a admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.393498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990254c1-b85d-4b3f-81bb-6763f6657bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.394330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990254c1-b85d-4b3f-81bb-6763f6657bae admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.394678 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990254c1-b85d-4b3f-81bb-6763f6657bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.395024 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990254c1-b85d-4b3f-81bb-6763f6657bae admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.395851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990254c1-b85d-4b3f-81bb-6763f6657bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.397529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990254c1-b85d-4b3f-81bb-6763f6657bae admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.398980 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c28d6-997e-4d53-9df2-b7ffee4d8f0a admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.403651 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99c9f2-e249-4ee0-94e1-6565bb0e2bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.404298 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99c9f2-e249-4ee0-94e1-6565bb0e2bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.404582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99c9f2-e249-4ee0-94e1-6565bb0e2bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.404860 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99c9f2-e249-4ee0-94e1-6565bb0e2bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.405594 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99c9f2-e249-4ee0-94e1-6565bb0e2bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.407195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99c9f2-e249-4ee0-94e1-6565bb0e2bb4 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.408556 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 793/1593] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:33.409362 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990254c1-b85d-4b3f-81bb-6763f6657bae admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.417714 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 801/1594] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:33.418568 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99c9f2-e249-4ee0-94e1-6565bb0e2bb4 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.420907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990254c1-b85d-4b3f-81bb-6763f6657bae admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.430957 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fd696-e56f-4147-9b08-e8c5df07f39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.432190 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fd696-e56f-4147-9b08-e8c5df07f39c admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.432328 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99c9f2-e249-4ee0-94e1-6565bb0e2bb4 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.432437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fd696-e56f-4147-9b08-e8c5df07f39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.432437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fd696-e56f-4147-9b08-e8c5df07f39c admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.433271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fd696-e56f-4147-9b08-e8c5df07f39c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.435204 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fd696-e56f-4147-9b08-e8c5df07f39c admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.436822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4db98c-9e51-4f17-bc0a-513c2fd01a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.437479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4db98c-9e51-4f17-bc0a-513c2fd01a9d admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.437766 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4db98c-9e51-4f17-bc0a-513c2fd01a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.438044 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4db98c-9e51-4f17-bc0a-513c2fd01a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.438808 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4db98c-9e51-4f17-bc0a-513c2fd01a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.440403 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4db98c-9e51-4f17-bc0a-513c2fd01a9d admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.445929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 794/1595] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:33.446904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fd696-e56f-4147-9b08-e8c5df07f39c admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.453725 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 802/1596] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:33.453725 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4db98c-9e51-4f17-bc0a-513c2fd01a9d admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.456857 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fd696-e56f-4147-9b08-e8c5df07f39c admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.461141 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4db98c-9e51-4f17-bc0a-513c2fd01a9d admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.461755 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98081f0-af31-4416-bcaa-970b7bd4e293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.462473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98081f0-af31-4416-bcaa-970b7bd4e293 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.462758 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98081f0-af31-4416-bcaa-970b7bd4e293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.463031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98081f0-af31-4416-bcaa-970b7bd4e293 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.463720 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98081f0-af31-4416-bcaa-970b7bd4e293 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.465726 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9eaf1-ed5b-4e01-ae10-f77b3dff0ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.465847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98081f0-af31-4416-bcaa-970b7bd4e293 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.466387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9eaf1-ed5b-4e01-ae10-f77b3dff0ddd admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.466673 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9eaf1-ed5b-4e01-ae10-f77b3dff0ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.466951 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9eaf1-ed5b-4e01-ae10-f77b3dff0ddd admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.467669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e9eaf1-ed5b-4e01-ae10-f77b3dff0ddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.469233 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e9eaf1-ed5b-4e01-ae10-f77b3dff0ddd admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.479423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 803/1597] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:33.480208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9eaf1-ed5b-4e01-ae10-f77b3dff0ddd admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.489962 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9eaf1-ed5b-4e01-ae10-f77b3dff0ddd admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.494482 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1c571-8eb9-4ec1-ba72-536b42b9547d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.495088 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1c571-8eb9-4ec1-ba72-536b42b9547d admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.495379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1c571-8eb9-4ec1-ba72-536b42b9547d admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.495657 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1c571-8eb9-4ec1-ba72-536b42b9547d admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.496367 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b1c571-8eb9-4ec1-ba72-536b42b9547d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.497918 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b1c571-8eb9-4ec1-ba72-536b42b9547d admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.507372 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 804/1598] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:33.508121 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1c571-8eb9-4ec1-ba72-536b42b9547d admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.517016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1c571-8eb9-4ec1-ba72-536b42b9547d admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.522243 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4967a-6e8f-46c7-a5fb-2925b0889957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.522849 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4967a-6e8f-46c7-a5fb-2925b0889957 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.523134 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4967a-6e8f-46c7-a5fb-2925b0889957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.523410 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4967a-6e8f-46c7-a5fb-2925b0889957 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.524113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4967a-6e8f-46c7-a5fb-2925b0889957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.525634 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4967a-6e8f-46c7-a5fb-2925b0889957 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.535432 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 805/1599] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:33.536758 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4967a-6e8f-46c7-a5fb-2925b0889957 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.545616 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4967a-6e8f-46c7-a5fb-2925b0889957 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.550507 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c9a65-15bf-46c5-9efb-ab6a91c8dee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.551157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c9a65-15bf-46c5-9efb-ab6a91c8dee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.551415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c9a65-15bf-46c5-9efb-ab6a91c8dee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.551740 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c9a65-15bf-46c5-9efb-ab6a91c8dee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.552489 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c9a65-15bf-46c5-9efb-ab6a91c8dee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.554044 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c9a65-15bf-46c5-9efb-ab6a91c8dee2 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.561738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 795/1600] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:16:33 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:33.561738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98081f0-af31-4416-bcaa-970b7bd4e293 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.567580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 806/1601] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:33.567580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c9a65-15bf-46c5-9efb-ab6a91c8dee2 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.569617 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98081f0-af31-4416-bcaa-970b7bd4e293 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.574232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce19903-6118-48c7-aba4-2fa1603a2671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.575109 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce19903-6118-48c7-aba4-2fa1603a2671 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.575395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce19903-6118-48c7-aba4-2fa1603a2671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.575672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce19903-6118-48c7-aba4-2fa1603a2671 admin admin] PATH_INFO: `/v3/projects/821dc2c3061041a3ab6ee939c306a7ea/users/35e69662bc7648b8b3e7467cf56c7f5e/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.576900 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c9a65-15bf-46c5-9efb-ab6a91c8dee2 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.582125 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069b845-bf79-46a7-a9ff-96db8bdd6a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.582775 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069b845-bf79-46a7-a9ff-96db8bdd6a00 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.583597 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069b845-bf79-46a7-a9ff-96db8bdd6a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.583994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069b845-bf79-46a7-a9ff-96db8bdd6a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.584734 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069b845-bf79-46a7-a9ff-96db8bdd6a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.586348 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069b845-bf79-46a7-a9ff-96db8bdd6a00 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.591166 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce19903-6118-48c7-aba4-2fa1603a2671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=821dc2c3061041a3ab6ee939c306a7ea, user_id=35e69662bc7648b8b3e7467cf56c7f5e, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.593783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce19903-6118-48c7-aba4-2fa1603a2671 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.597082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 807/1602] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:33.598282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069b845-bf79-46a7-a9ff-96db8bdd6a00 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.608812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069b845-bf79-46a7-a9ff-96db8bdd6a00 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.613637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ebf28-2bcd-488f-b4ff-203f86fcffa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.614587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ebf28-2bcd-488f-b4ff-203f86fcffa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.614920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ebf28-2bcd-488f-b4ff-203f86fcffa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.615145 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ebf28-2bcd-488f-b4ff-203f86fcffa0 admin admin] PATH_INFO: `/v3/projects/9d38de3da9e3489ebbbebd4f8aeb0d49/users/261d01f5b22340a09708c71be64edae8/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.629160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ebf28-2bcd-488f-b4ff-203f86fcffa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d38de3da9e3489ebbbebd4f8aeb0d49, user_id=261d01f5b22340a09708c71be64edae8, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.631978 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ebf28-2bcd-488f-b4ff-203f86fcffa0 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.667976 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 796/1603] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:33 2022] PUT /identity/v3/projects/821dc2c3061041a3ab6ee939c306a7ea/users/35e69662bc7648b8b3e7467cf56c7f5e/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:33.668885 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce19903-6118-48c7-aba4-2fa1603a2671 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.695676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 808/1604] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:33 2022] PUT /identity/v3/projects/9d38de3da9e3489ebbbebd4f8aeb0d49/users/261d01f5b22340a09708c71be64edae8/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:33.696550 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ebf28-2bcd-488f-b4ff-203f86fcffa0 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.731784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce19903-6118-48c7-aba4-2fa1603a2671 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.756349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ebf28-2bcd-488f-b4ff-203f86fcffa0 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.765799 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6d42d-eaca-469e-a924-dea4dfd02a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.766822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6d42d-eaca-469e-a924-dea4dfd02a1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.767175 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6d42d-eaca-469e-a924-dea4dfd02a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.767561 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6d42d-eaca-469e-a924-dea4dfd02a1f admin admin] PATH_INFO: `/v3/projects/d5cecfb8eb5e4b8ea121c2ef8a9c28a2/users/8feebf7f09cb4f7d81759f8f1336dba5/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.784475 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6d42d-eaca-469e-a924-dea4dfd02a1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5cecfb8eb5e4b8ea121c2ef8a9c28a2, user_id=8feebf7f09cb4f7d81759f8f1336dba5, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.787868 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6d42d-eaca-469e-a924-dea4dfd02a1f admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.788499 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6f997-b34e-4fe9-bdc5-7a120f57c155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.789538 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6f997-b34e-4fe9-bdc5-7a120f57c155 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.789822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6f997-b34e-4fe9-bdc5-7a120f57c155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.790123 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6f997-b34e-4fe9-bdc5-7a120f57c155 admin admin] PATH_INFO: `/v3/projects/19fcef97d9fc42889ca1435f33cef96b/users/c9bc1ceb41a6485094ece96346398ef0/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.804813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6f997-b34e-4fe9-bdc5-7a120f57c155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19fcef97d9fc42889ca1435f33cef96b, user_id=c9bc1ceb41a6485094ece96346398ef0, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.807536 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6f997-b34e-4fe9-bdc5-7a120f57c155 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.884492 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 809/1605] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:33 2022] PUT /identity/v3/projects/d5cecfb8eb5e4b8ea121c2ef8a9c28a2/users/8feebf7f09cb4f7d81759f8f1336dba5/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:33.888560 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 797/1606] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:33 2022] PUT /identity/v3/projects/19fcef97d9fc42889ca1435f33cef96b/users/c9bc1ceb41a6485094ece96346398ef0/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:33.888682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6d42d-eaca-469e-a924-dea4dfd02a1f admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.888791 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6f997-b34e-4fe9-bdc5-7a120f57c155 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.949421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6f997-b34e-4fe9-bdc5-7a120f57c155 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.949670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6d42d-eaca-469e-a924-dea4dfd02a1f admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.954283 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18628c-afff-4706-9ec6-c6721029b4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.954629 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155223e8-d74a-4083-985e-00cda7517591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.954949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18628c-afff-4706-9ec6-c6721029b4bf admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.955235 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18628c-afff-4706-9ec6-c6721029b4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.955514 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18628c-afff-4706-9ec6-c6721029b4bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.955625 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155223e8-d74a-4083-985e-00cda7517591 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.955837 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155223e8-d74a-4083-985e-00cda7517591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.956113 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155223e8-d74a-4083-985e-00cda7517591 admin admin] PATH_INFO: `/v3/projects/2969433bca8d4ef494ac0cffe63f962d/users/b99b106f2f7f47eea7dbf145d7cc1b31/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.956244 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18628c-afff-4706-9ec6-c6721029b4bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.957841 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18628c-afff-4706-9ec6-c6721029b4bf admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.968112 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 810/1607] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:33.968891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18628c-afff-4706-9ec6-c6721029b4bf admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:33.971490 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155223e8-d74a-4083-985e-00cda7517591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2969433bca8d4ef494ac0cffe63f962d, user_id=b99b106f2f7f47eea7dbf145d7cc1b31, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:33.974250 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155223e8-d74a-4083-985e-00cda7517591 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:33.978414 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18628c-afff-4706-9ec6-c6721029b4bf admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:33.983279 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76eb69-00be-469e-a0da-cc2b34313e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:33.984144 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76eb69-00be-469e-a0da-cc2b34313e77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:33.984432 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76eb69-00be-469e-a0da-cc2b34313e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:33.984738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76eb69-00be-469e-a0da-cc2b34313e77 admin admin] PATH_INFO: `/v3/projects/d3967d950ee74522aeee6b82e4b4615a/users/56be7d9ead0f4e1f8d12c29e62e40e1f/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:33.999085 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76eb69-00be-469e-a0da-cc2b34313e77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3967d950ee74522aeee6b82e4b4615a, user_id=56be7d9ead0f4e1f8d12c29e62e40e1f, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:34.001710 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76eb69-00be-469e-a0da-cc2b34313e77 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:34.061187 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 811/1608] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:33 2022] PUT /identity/v3/projects/d3967d950ee74522aeee6b82e4b4615a/users/56be7d9ead0f4e1f8d12c29e62e40e1f/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:34.061510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 798/1609] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:33 2022] PUT /identity/v3/projects/2969433bca8d4ef494ac0cffe63f962d/users/b99b106f2f7f47eea7dbf145d7cc1b31/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:34.062982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5188085-640f-485d-8045-027d8b45bf0e None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.063271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5188085-640f-485d-8045-027d8b45bf0e None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.063548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5188085-640f-485d-8045-027d8b45bf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.072177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951152f0-d18f-4af5-8c4a-2518dfa85f89 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.072472 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951152f0-d18f-4af5-8c4a-2518dfa85f89 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.072752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951152f0-d18f-4af5-8c4a-2518dfa85f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.088532 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-c5188085-640f-485d-8045-027d8b45bf0e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:34.089737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c5188085-640f-485d-8045-027d8b45bf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:34.090256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-951152f0-d18f-4af5-8c4a-2518dfa85f89 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:34.091429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-951152f0-d18f-4af5-8c4a-2518dfa85f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:34.146490 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 799/1610] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:34.147776 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c61ef-1106-457a-b7a1-064c30eb7965 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.148066 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c61ef-1106-457a-b7a1-064c30eb7965 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.148353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c61ef-1106-457a-b7a1-064c30eb7965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.156848 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 812/1611] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:34.158276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68380f-b64e-4710-bd09-22b32540cd11 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.158641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68380f-b64e-4710-bd09-22b32540cd11 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.158996 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68380f-b64e-4710-bd09-22b32540cd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.167557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-733c61ef-1106-457a-b7a1-064c30eb7965 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:34.168763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-733c61ef-1106-457a-b7a1-064c30eb7965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:34.176921 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 800/1612] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:34.177059 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-7a68380f-b64e-4710-bd09-22b32540cd11 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:34.177671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c61ef-1106-457a-b7a1-064c30eb7965 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:34.178345 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7a68380f-b64e-4710-bd09-22b32540cd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:34.187336 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c61ef-1106-457a-b7a1-064c30eb7965 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:34.187782 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 813/1613] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:34.189084 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d081007-f178-41ec-a5fc-e241a92fa1c8 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.189381 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d081007-f178-41ec-a5fc-e241a92fa1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.189664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d081007-f178-41ec-a5fc-e241a92fa1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.195914 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08a5c9-9808-408c-8ced-f79148ff40dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:34.197404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08a5c9-9808-408c-8ced-f79148ff40dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.197404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08a5c9-9808-408c-8ced-f79148ff40dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.197404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08a5c9-9808-408c-8ced-f79148ff40dd admin admin] PATH_INFO: `/v3/projects/52768df123974dccb42a0b088dabd99a/users/62487501d4364e5284c8ed829f9223f0/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.212361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d08a5c9-9808-408c-8ced-f79148ff40dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=52768df123974dccb42a0b088dabd99a, user_id=62487501d4364e5284c8ed829f9223f0, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:34.215036 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d08a5c9-9808-408c-8ced-f79148ff40dd admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:34.216540 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-4d081007-f178-41ec-a5fc-e241a92fa1c8 None None] MFA Rules not processed for user `56be7d9ead0f4e1f8d12c29e62e40e1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:34.217744 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4d081007-f178-41ec-a5fc-e241a92fa1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:34.278711 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 801/1614] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:34 2022] PUT /identity/v3/projects/52768df123974dccb42a0b088dabd99a/users/62487501d4364e5284c8ed829f9223f0/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:34.279577 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 814/1615] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:34 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:34.280057 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2306ba-e7b2-4ce7-ba76-e4c6b2c5217c None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.280760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d081007-f178-41ec-a5fc-e241a92fa1c8 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:34.281089 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2306ba-e7b2-4ce7-ba76-e4c6b2c5217c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.281390 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2306ba-e7b2-4ce7-ba76-e4c6b2c5217c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.307845 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-5f2306ba-e7b2-4ce7-ba76-e4c6b2c5217c None None] MFA Rules not processed for user `b99b106f2f7f47eea7dbf145d7cc1b31`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:34.309099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-5f2306ba-e7b2-4ce7-ba76-e4c6b2c5217c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:34.340590 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d081007-f178-41ec-a5fc-e241a92fa1c8 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:34.373080 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 802/1616] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:34.374056 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2306ba-e7b2-4ce7-ba76-e4c6b2c5217c None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:34.387364 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6dc05c-0dfa-471b-859b-b0a9f0724ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:34.388010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dc05c-0dfa-471b-859b-b0a9f0724ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.388291 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dc05c-0dfa-471b-859b-b0a9f0724ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.388566 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dc05c-0dfa-471b-859b-b0a9f0724ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.432154 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2306ba-e7b2-4ce7-ba76-e4c6b2c5217c None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:34.435856 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4cf7a-1345-48f5-8789-24f9cc6a0cbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:34.436515 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4cf7a-1345-48f5-8789-24f9cc6a0cbe service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.436800 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4cf7a-1345-48f5-8789-24f9cc6a0cbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.437079 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4cf7a-1345-48f5-8789-24f9cc6a0cbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.447890 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6dc05c-0dfa-471b-859b-b0a9f0724ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:34.449337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6dc05c-0dfa-471b-859b-b0a9f0724ba4 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:34.458781 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 815/1617] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:34.459565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6dc05c-0dfa-471b-859b-b0a9f0724ba4 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:34.466929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6dc05c-0dfa-471b-859b-b0a9f0724ba4 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:34.471201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf263e5-7ac6-4c93-8979-1e05dd1af132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:34.471842 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf263e5-7ac6-4c93-8979-1e05dd1af132 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.472126 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf263e5-7ac6-4c93-8979-1e05dd1af132 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.472410 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf263e5-7ac6-4c93-8979-1e05dd1af132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.483257 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf263e5-7ac6-4c93-8979-1e05dd1af132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:34.484900 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf263e5-7ac6-4c93-8979-1e05dd1af132 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:34.495401 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 816/1618] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:34.498942 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4cf7a-1345-48f5-8789-24f9cc6a0cbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:34.500389 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4cf7a-1345-48f5-8789-24f9cc6a0cbe service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:34.503085 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf263e5-7ac6-4c93-8979-1e05dd1af132 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:34.511467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 803/1619] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:34.512219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4cf7a-1345-48f5-8789-24f9cc6a0cbe service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:34.512927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf263e5-7ac6-4c93-8979-1e05dd1af132 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:34.517182 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8ab0d-24b6-448b-8b48-ff11264d0850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:34.518168 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8ab0d-24b6-448b-8b48-ff11264d0850 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.518565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8ab0d-24b6-448b-8b48-ff11264d0850 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.518953 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8ab0d-24b6-448b-8b48-ff11264d0850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.521796 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4cf7a-1345-48f5-8789-24f9cc6a0cbe service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:34.526618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99dd1b-d912-4dd4-8205-01f21a9f9121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:34.527499 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99dd1b-d912-4dd4-8205-01f21a9f9121 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.527629 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99dd1b-d912-4dd4-8205-01f21a9f9121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.527834 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99dd1b-d912-4dd4-8205-01f21a9f9121 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.528713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99dd1b-d912-4dd4-8205-01f21a9f9121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:34.529986 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8ab0d-24b6-448b-8b48-ff11264d0850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:34.530365 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99dd1b-d912-4dd4-8205-01f21a9f9121 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:34.531714 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8ab0d-24b6-448b-8b48-ff11264d0850 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:34.544754 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 804/1620] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:34.545094 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 817/1621] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:34.545192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99dd1b-d912-4dd4-8205-01f21a9f9121 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:34.545297 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8ab0d-24b6-448b-8b48-ff11264d0850 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:34.554216 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8ab0d-24b6-448b-8b48-ff11264d0850 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:34.555942 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99dd1b-d912-4dd4-8205-01f21a9f9121 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:34.559296 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3850a5-dcd5-42cf-a5b8-d14d12958c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:34.560056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3850a5-dcd5-42cf-a5b8-d14d12958c9c admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.560489 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3850a5-dcd5-42cf-a5b8-d14d12958c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.560865 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3850a5-dcd5-42cf-a5b8-d14d12958c9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.560981 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea1e2d-691f-46b0-a2d8-0ed03279a559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:34.561539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea1e2d-691f-46b0-a2d8-0ed03279a559 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.561823 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea1e2d-691f-46b0-a2d8-0ed03279a559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.561946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3850a5-dcd5-42cf-a5b8-d14d12958c9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:34.562212 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea1e2d-691f-46b0-a2d8-0ed03279a559 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.563096 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea1e2d-691f-46b0-a2d8-0ed03279a559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:34.564403 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3850a5-dcd5-42cf-a5b8-d14d12958c9c admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:34.564847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea1e2d-691f-46b0-a2d8-0ed03279a559 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:34.675270 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 805/1622] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:16:34 2022] POST /identity/v3/projects => generated 355 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:34.676103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea1e2d-691f-46b0-a2d8-0ed03279a559 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:34.681099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 818/1623] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:16:34 2022] POST /identity/v3/projects => generated 405 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:34.682392 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3850a5-dcd5-42cf-a5b8-d14d12958c9c admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:34.750230 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3850a5-dcd5-42cf-a5b8-d14d12958c9c admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:34.751236 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea1e2d-691f-46b0-a2d8-0ed03279a559 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:34.755623 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7056f-584f-4d86-b56e-079cdcc3ec84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:34.756411 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7056f-584f-4d86-b56e-079cdcc3ec84 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.756865 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7056f-584f-4d86-b56e-079cdcc3ec84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.756999 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781e47c-9aca-4673-aa6d-6630c6a0391f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:34.757354 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7056f-584f-4d86-b56e-079cdcc3ec84 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.757815 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781e47c-9aca-4673-aa6d-6630c6a0391f admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.758107 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781e47c-9aca-4673-aa6d-6630c6a0391f admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.758386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781e47c-9aca-4673-aa6d-6630c6a0391f admin admin] PATH_INFO: `/v3/projects/52768df123974dccb42a0b088dabd99a/users/62487501d4364e5284c8ed829f9223f0/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.758493 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa7056f-584f-4d86-b56e-079cdcc3ec84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:34.760695 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa7056f-584f-4d86-b56e-079cdcc3ec84 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:34.760944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781e47c-9aca-4673-aa6d-6630c6a0391f admin admin] RBAC: Authorizing `identity:create_grant(project_id=52768df123974dccb42a0b088dabd99a, user_id=62487501d4364e5284c8ed829f9223f0, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:34.764506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781e47c-9aca-4673-aa6d-6630c6a0391f admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:34.776152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 806/1624] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:34 2022] PUT /identity/v3/projects/52768df123974dccb42a0b088dabd99a/users/62487501d4364e5284c8ed829f9223f0/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:34.777098 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781e47c-9aca-4673-aa6d-6630c6a0391f admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:34.841091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781e47c-9aca-4673-aa6d-6630c6a0391f admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:34.846541 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaaffd-f08e-482c-898d-c4127637ff20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:34.847207 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaaffd-f08e-482c-898d-c4127637ff20 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.847490 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaaffd-f08e-482c-898d-c4127637ff20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.847762 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaaffd-f08e-482c-898d-c4127637ff20 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.849038 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aaaffd-f08e-482c-898d-c4127637ff20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:34.851009 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aaaffd-f08e-482c-898d-c4127637ff20 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:34.868125 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 819/1625] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:16:34 2022] POST /identity/v3/users => generated 284 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:34.869295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b326b2-ea3e-4542-9a3a-d847158d0034 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.871301 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b326b2-ea3e-4542-9a3a-d847158d0034 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.871301 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b326b2-ea3e-4542-9a3a-d847158d0034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.900649 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-c8b326b2-ea3e-4542-9a3a-d847158d0034 None None] MFA Rules not processed for user `62487501d4364e5284c8ed829f9223f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:34.902411 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-c8b326b2-ea3e-4542-9a3a-d847158d0034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:34.944362 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 807/1626] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:16:34 2022] POST /identity/v3/users => generated 309 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:34.945817 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaaffd-f08e-482c-898d-c4127637ff20 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:34.959793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaaffd-f08e-482c-898d-c4127637ff20 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:34.967607 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a25ec-88f2-41a8-900c-2ccdb7043cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:34.969072 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a25ec-88f2-41a8-900c-2ccdb7043cdd admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:34.969526 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a25ec-88f2-41a8-900c-2ccdb7043cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:34.970035 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a25ec-88f2-41a8-900c-2ccdb7043cdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:34.972015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a25ec-88f2-41a8-900c-2ccdb7043cdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:34.975966 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a25ec-88f2-41a8-900c-2ccdb7043cdd admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:34.994682 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 808/1627] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:34.995685 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a25ec-88f2-41a8-900c-2ccdb7043cdd admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:35.009761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a25ec-88f2-41a8-900c-2ccdb7043cdd admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:35.029168 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 820/1628] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:35.029168 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b326b2-ea3e-4542-9a3a-d847158d0034 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:35.029557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c7a15-a571-44df-8dfa-bfcde2fa8c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:35.029557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c7a15-a571-44df-8dfa-bfcde2fa8c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:35.029557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c7a15-a571-44df-8dfa-bfcde2fa8c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:35.029557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c7a15-a571-44df-8dfa-bfcde2fa8c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:35.029899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c7a15-a571-44df-8dfa-bfcde2fa8c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:35.029899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c7a15-a571-44df-8dfa-bfcde2fa8c9b admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:35.033393 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b326b2-ea3e-4542-9a3a-d847158d0034 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:35.039411 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b30da0-f1e2-401a-b8c9-cad5faa8ebe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:35.041420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b30da0-f1e2-401a-b8c9-cad5faa8ebe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:35.041777 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b30da0-f1e2-401a-b8c9-cad5faa8ebe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:35.042053 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b30da0-f1e2-401a-b8c9-cad5faa8ebe3 admin admin] PATH_INFO: `/v3/projects/f102c3f973aa49729ab0898bb28cd3bd/users/2a72b372c8d0469e815fbe536b4cf9bb/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:35.049055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 809/1629] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:35.053132 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c7a15-a571-44df-8dfa-bfcde2fa8c9b admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:35.068974 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b30da0-f1e2-401a-b8c9-cad5faa8ebe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f102c3f973aa49729ab0898bb28cd3bd, user_id=2a72b372c8d0469e815fbe536b4cf9bb, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:35.072519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b30da0-f1e2-401a-b8c9-cad5faa8ebe3 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:35.119936 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c7a15-a571-44df-8dfa-bfcde2fa8c9b admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:35.125223 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f972054-ffaf-425f-a434-e993d79ed951 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:35.126165 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f972054-ffaf-425f-a434-e993d79ed951 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:35.131289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f972054-ffaf-425f-a434-e993d79ed951 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:35.131289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f972054-ffaf-425f-a434-e993d79ed951 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:35.141616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f972054-ffaf-425f-a434-e993d79ed951 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:35.143229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f972054-ffaf-425f-a434-e993d79ed951 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:35.154402 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 821/1630] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:35 2022] PUT /identity/v3/projects/f102c3f973aa49729ab0898bb28cd3bd/users/2a72b372c8d0469e815fbe536b4cf9bb/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:35.156044 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b30da0-f1e2-401a-b8c9-cad5faa8ebe3 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:35.165645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 810/1631] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:35 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:35.167664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f972054-ffaf-425f-a434-e993d79ed951 service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:35.183207 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f972054-ffaf-425f-a434-e993d79ed951 service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:35.239280 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b30da0-f1e2-401a-b8c9-cad5faa8ebe3 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:35.245464 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dccb59-36dc-42cf-8610-639d15e21d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:35.246562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dccb59-36dc-42cf-8610-639d15e21d4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:35.246953 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dccb59-36dc-42cf-8610-639d15e21d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:35.247877 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dccb59-36dc-42cf-8610-639d15e21d4b admin admin] PATH_INFO: `/v3/projects/a023f221df7c45c1b166c8b63a0d7963/users/2612dcd40b8641f58cbeb186fc579974/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:35.258043 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3dba92-e734-4b68-9bcd-31252ffd1ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:35.270006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dba92-e734-4b68-9bcd-31252ffd1ebb admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:35.270897 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dba92-e734-4b68-9bcd-31252ffd1ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:35.272664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dba92-e734-4b68-9bcd-31252ffd1ebb admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:35.274413 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dccb59-36dc-42cf-8610-639d15e21d4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a023f221df7c45c1b166c8b63a0d7963, user_id=2612dcd40b8641f58cbeb186fc579974, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:35.274750 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3dba92-e734-4b68-9bcd-31252ffd1ebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:35.276404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3dba92-e734-4b68-9bcd-31252ffd1ebb admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:35.278356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dccb59-36dc-42cf-8610-639d15e21d4b admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:35.291879 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 811/1632] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:35.298007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3dba92-e734-4b68-9bcd-31252ffd1ebb admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:35.311292 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3dba92-e734-4b68-9bcd-31252ffd1ebb admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:35.316850 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3a27e-7634-47e8-82e9-6703d9a9709b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:35.317871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3a27e-7634-47e8-82e9-6703d9a9709b admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:35.318275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3a27e-7634-47e8-82e9-6703d9a9709b admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:35.318649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3a27e-7634-47e8-82e9-6703d9a9709b admin admin] PATH_INFO: `/v3/projects/f102c3f973aa49729ab0898bb28cd3bd/users/2a72b372c8d0469e815fbe536b4cf9bb/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:35.320801 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa3a27e-7634-47e8-82e9-6703d9a9709b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f102c3f973aa49729ab0898bb28cd3bd, user_id=2a72b372c8d0469e815fbe536b4cf9bb, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:35.324253 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa3a27e-7634-47e8-82e9-6703d9a9709b admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:35.335728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 812/1633] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:35 2022] PUT /identity/v3/projects/f102c3f973aa49729ab0898bb28cd3bd/users/2a72b372c8d0469e815fbe536b4cf9bb/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:16:35.342382 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c166c7-cb48-403c-816a-21d09809f84f None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:35.342382 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c166c7-cb48-403c-816a-21d09809f84f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:35.342382 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c166c7-cb48-403c-816a-21d09809f84f None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:35.342382 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 813/1634] 10.222.0.65 () {54 vars in 961 bytes} [Tue May 10 20:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:16:35.346129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404aa2dd-e9a9-4c56-ae93-b726620e3635 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:35.346451 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404aa2dd-e9a9-4c56-ae93-b726620e3635 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:35.346733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404aa2dd-e9a9-4c56-ae93-b726620e3635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:35.365824 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-404aa2dd-e9a9-4c56-ae93-b726620e3635 None None] MFA Rules not processed for user `ae922b2da4814d10bed1fa345700ac04`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:35.367288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-404aa2dd-e9a9-4c56-ae93-b726620e3635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:35.370506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 822/1635] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:35 2022] PUT /identity/v3/projects/a023f221df7c45c1b166c8b63a0d7963/users/2612dcd40b8641f58cbeb186fc579974/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:35.371761 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d17b8-75c2-408b-b199-fb50196cab8b None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:35.372044 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d17b8-75c2-408b-b199-fb50196cab8b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:35.372315 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d17b8-75c2-408b-b199-fb50196cab8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:35.400667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-e25d17b8-75c2-408b-b199-fb50196cab8b None None] MFA Rules not processed for user `2a72b372c8d0469e815fbe536b4cf9bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:35.406272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-e25d17b8-75c2-408b-b199-fb50196cab8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:35.484194 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 814/1636] 10.222.0.65 () {58 vars in 1058 bytes} [Tue May 10 20:16:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:16:35.485210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404aa2dd-e9a9-4c56-ae93-b726620e3635 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:35.505327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 823/1637] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:35 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:35.506347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d17b8-75c2-408b-b199-fb50196cab8b None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:35.584903 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404aa2dd-e9a9-4c56-ae93-b726620e3635 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:35.591282 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6fa80-be45-4c08-8817-8dc1171602c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:35.591973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6fa80-be45-4c08-8817-8dc1171602c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:35.593164 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d17b8-75c2-408b-b199-fb50196cab8b None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:35.593718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6fa80-be45-4c08-8817-8dc1171602c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:35.594009 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6fa80-be45-4c08-8817-8dc1171602c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:35.595008 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6fa80-be45-4c08-8817-8dc1171602c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:35.597886 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6fa80-be45-4c08-8817-8dc1171602c7 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:35.598924 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a87e8-7b10-454c-85c6-21118a5ae62e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:35.599912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a87e8-7b10-454c-85c6-21118a5ae62e service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:35.605749 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a87e8-7b10-454c-85c6-21118a5ae62e service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:35.605749 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a87e8-7b10-454c-85c6-21118a5ae62e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:35.611045 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 815/1638] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:35.612561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ceb81c-6954-4b76-9d34-607f21ca9ad3 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:35.612854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ceb81c-6954-4b76-9d34-607f21ca9ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:35.613134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ceb81c-6954-4b76-9d34-607f21ca9ad3 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:35.621751 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 816/1639] 10.222.0.65 () {54 vars in 912 bytes} [Tue May 10 20:16:35 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:16:35.622867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ceb81c-6954-4b76-9d34-607f21ca9ad3 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:35.642338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ceb81c-6954-4b76-9d34-607f21ca9ad3 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:35.656044 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ecd9f-5aec-4251-bf08-da1833da5d35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:35.656779 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ecd9f-5aec-4251-bf08-da1833da5d35 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:35.657199 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ecd9f-5aec-4251-bf08-da1833da5d35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:35.657503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ecd9f-5aec-4251-bf08-da1833da5d35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:35.674890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ecd9f-5aec-4251-bf08-da1833da5d35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:35.680386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ecd9f-5aec-4251-bf08-da1833da5d35 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:35.688308 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 817/1640] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:35 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:35.689298 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ecd9f-5aec-4251-bf08-da1833da5d35 service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:35.701471 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ecd9f-5aec-4251-bf08-da1833da5d35 service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:35.707096 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd48c95-31d6-4fb1-b192-1f39fada1853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:35.707787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd48c95-31d6-4fb1-b192-1f39fada1853 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:35.708671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd48c95-31d6-4fb1-b192-1f39fada1853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:35.709078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd48c95-31d6-4fb1-b192-1f39fada1853 admin admin] PATH_INFO: `/v3/projects/a023f221df7c45c1b166c8b63a0d7963/users/2612dcd40b8641f58cbeb186fc579974/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:35.711480 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd48c95-31d6-4fb1-b192-1f39fada1853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a023f221df7c45c1b166c8b63a0d7963, user_id=2612dcd40b8641f58cbeb186fc579974, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:35.713370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a87e8-7b10-454c-85c6-21118a5ae62e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:35.714538 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd48c95-31d6-4fb1-b192-1f39fada1853 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:35.717254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a87e8-7b10-454c-85c6-21118a5ae62e service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:35.737775 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 818/1641] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:35 2022] PUT /identity/v3/projects/a023f221df7c45c1b166c8b63a0d7963/users/2612dcd40b8641f58cbeb186fc579974/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:35.737775 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e46dd3-c8f1-46dd-a9bf-33bb2ff4a403 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:35.737775 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e46dd3-c8f1-46dd-a9bf-33bb2ff4a403 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:35.737775 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e46dd3-c8f1-46dd-a9bf-33bb2ff4a403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:35.766565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 824/1642] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:35.769053 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f42546-9368-4af5-9713-f146b9a50106 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:35.769431 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f42546-9368-4af5-9713-f146b9a50106 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:35.769724 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f42546-9368-4af5-9713-f146b9a50106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:35.780482 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-50e46dd3-c8f1-46dd-a9bf-33bb2ff4a403 None None] MFA Rules not processed for user `ae922b2da4814d10bed1fa345700ac04`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:35.782346 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-50e46dd3-c8f1-46dd-a9bf-33bb2ff4a403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:35.830893 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-a7f42546-9368-4af5-9713-f146b9a50106 None None] MFA Rules not processed for user `2612dcd40b8641f58cbeb186fc579974`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:35.832827 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a7f42546-9368-4af5-9713-f146b9a50106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:35.909872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 819/1643] 10.222.0.65 () {58 vars in 1009 bytes} [Tue May 10 20:16:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:35.928483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 825/1644] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:35 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:35.980764 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e46dd3-c8f1-46dd-a9bf-33bb2ff4a403 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:36.053435 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e46dd3-c8f1-46dd-a9bf-33bb2ff4a403 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:36.057363 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405f33d-8ffb-4cd0-908b-7b9c58a3b1c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:36.058013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405f33d-8ffb-4cd0-908b-7b9c58a3b1c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:36.058336 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405f33d-8ffb-4cd0-908b-7b9c58a3b1c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:36.058617 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405f33d-8ffb-4cd0-908b-7b9c58a3b1c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:36.070009 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a405f33d-8ffb-4cd0-908b-7b9c58a3b1c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:36.073081 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a405f33d-8ffb-4cd0-908b-7b9c58a3b1c3 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:36.082628 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 820/1645] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:36.136834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f42546-9368-4af5-9713-f146b9a50106 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:36.145945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f42546-9368-4af5-9713-f146b9a50106 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:36.149549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b616e-ab34-4126-88c0-066c8d1d2344 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:36.150240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b616e-ab34-4126-88c0-066c8d1d2344 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:36.155571 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b616e-ab34-4126-88c0-066c8d1d2344 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:36.155571 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b616e-ab34-4126-88c0-066c8d1d2344 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:36.160364 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b616e-ab34-4126-88c0-066c8d1d2344 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:36.162347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b616e-ab34-4126-88c0-066c8d1d2344 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:36.171915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 826/1646] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:36 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:36.270043 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405f33d-8ffb-4cd0-908b-7b9c58a3b1c3 service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:36.285810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405f33d-8ffb-4cd0-908b-7b9c58a3b1c3 service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:36.294957 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298c91c-ffe8-4c0a-a491-ae5982666b93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:36.305179 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298c91c-ffe8-4c0a-a491-ae5982666b93 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:36.305872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298c91c-ffe8-4c0a-a491-ae5982666b93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:36.306425 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298c91c-ffe8-4c0a-a491-ae5982666b93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:36.454583 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298c91c-ffe8-4c0a-a491-ae5982666b93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:36.457767 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298c91c-ffe8-4c0a-a491-ae5982666b93 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:36.476417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 821/1647] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:36 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:37.177683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b616e-ab34-4126-88c0-066c8d1d2344 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:37.240505 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e8fd9-dcef-4a7b-8899-44253e418031 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:37.241265 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e8fd9-dcef-4a7b-8899-44253e418031 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:37.241902 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e8fd9-dcef-4a7b-8899-44253e418031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:37.313481 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b616e-ab34-4126-88c0-066c8d1d2344 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:37.321659 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-4e7e8fd9-dcef-4a7b-8899-44253e418031 None None] MFA Rules not processed for user `c9bc1ceb41a6485094ece96346398ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:37.323780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-4e7e8fd9-dcef-4a7b-8899-44253e418031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:37.330527 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0cb44c-0fae-48da-ab35-531d2a95ab65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:37.332058 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0cb44c-0fae-48da-ab35-531d2a95ab65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:37.333506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0cb44c-0fae-48da-ab35-531d2a95ab65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:37.334209 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0cb44c-0fae-48da-ab35-531d2a95ab65 admin admin] PATH_INFO: `/v3/users/b99b106f2f7f47eea7dbf145d7cc1b31` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:37.337405 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0cb44c-0fae-48da-ab35-531d2a95ab65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b99b106f2f7f47eea7dbf145d7cc1b31)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:37.342459 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0cb44c-0fae-48da-ab35-531d2a95ab65 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:37.445904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 822/1648] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:37.458813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c2ebb-e19d-40ef-85e2-486f8db0cc8d None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:37.463117 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c2ebb-e19d-40ef-85e2-486f8db0cc8d None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:37.463117 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c2ebb-e19d-40ef-85e2-486f8db0cc8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:37.486142 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-d67c2ebb-e19d-40ef-85e2-486f8db0cc8d None None] MFA Rules not processed for user `c9bc1ceb41a6485094ece96346398ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:37.488223 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-d67c2ebb-e19d-40ef-85e2-486f8db0cc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:37.498470 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 823/1649] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:37.517824 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5580239-583c-496b-bc54-418c0c4a2871 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:37.518650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5580239-583c-496b-bc54-418c0c4a2871 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:37.519147 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5580239-583c-496b-bc54-418c0c4a2871 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:37.522071 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 824/1650] 10.222.0.65 () {54 vars in 961 bytes} [Tue May 10 20:16:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 20:16:37.532077 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608e18c-9fd8-48c6-9124-18ef440a0316 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:37.532186 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608e18c-9fd8-48c6-9124-18ef440a0316 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:37.532417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608e18c-9fd8-48c6-9124-18ef440a0316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:37.552097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-7608e18c-9fd8-48c6-9124-18ef440a0316 None None] MFA Rules not processed for user `ae922b2da4814d10bed1fa345700ac04`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:37.553290 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-7608e18c-9fd8-48c6-9124-18ef440a0316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:37.578239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 825/1651] 10.222.0.65 () {58 vars in 1058 bytes} [Tue May 10 20:16:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:16:37.579246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608e18c-9fd8-48c6-9124-18ef440a0316 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:37.581729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ce0cb44c-0fae-48da-ab35-531d2a95ab65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b99b106f2f7f47eea7dbf145d7cc1b31'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:37.583204 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ce0cb44c-0fae-48da-ab35-531d2a95ab65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b99b106f2f7f47eea7dbf145d7cc1b31'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:37.584253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ce0cb44c-0fae-48da-ab35-531d2a95ab65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b99b106f2f7f47eea7dbf145d7cc1b31'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:37.591569 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608e18c-9fd8-48c6-9124-18ef440a0316 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:37.596892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1d3d5-9f56-417c-9fbc-80d29e44c9c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:37.597673 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1d3d5-9f56-417c-9fbc-80d29e44c9c1 service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:37.597958 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1d3d5-9f56-417c-9fbc-80d29e44c9c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:37.599011 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1d3d5-9f56-417c-9fbc-80d29e44c9c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:37.601180 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8fc1d3d5-9f56-417c-9fbc-80d29e44c9c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:37.614166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ce0cb44c-0fae-48da-ab35-531d2a95ab65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b99b106f2f7f47eea7dbf145d7cc1b31'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:37.615246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1d3d5-9f56-417c-9fbc-80d29e44c9c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:37.618245 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1d3d5-9f56-417c-9fbc-80d29e44c9c1 service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:37.624557 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ce0cb44c-0fae-48da-ab35-531d2a95ab65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b99b106f2f7f47eea7dbf145d7cc1b31'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:37.638349 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 826/1652] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:37.726248 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 827/1653] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:16:37 2022] DELETE /identity/v3/users/b99b106f2f7f47eea7dbf145d7cc1b31 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:37.730717 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b34954-b592-4a31-9a45-652cdf823c79 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:37.731094 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b34954-b592-4a31-9a45-652cdf823c79 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:37.731416 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b34954-b592-4a31-9a45-652cdf823c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:37.769668 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-f5b34954-b592-4a31-9a45-652cdf823c79 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:37.775173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f5b34954-b592-4a31-9a45-652cdf823c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:37.868838 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 827/1654] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:37.877770 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0cb44c-0fae-48da-ab35-531d2a95ab65 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:37.881739 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-ce0cb44c-0fae-48da-ab35-531d2a95ab65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:38.061395 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0cb44c-0fae-48da-ab35-531d2a95ab65 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:38.079110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160be099-bbe2-41d7-abac-b4f0a951f246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:38.080207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160be099-bbe2-41d7-abac-b4f0a951f246 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:38.080493 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160be099-bbe2-41d7-abac-b4f0a951f246 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:38.080776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160be099-bbe2-41d7-abac-b4f0a951f246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:38.100399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160be099-bbe2-41d7-abac-b4f0a951f246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:38.101848 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160be099-bbe2-41d7-abac-b4f0a951f246 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:38.111601 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23b801-49e8-42b6-a8b9-e2041c3c0d24 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:38.128780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23b801-49e8-42b6-a8b9-e2041c3c0d24 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:38.128780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23b801-49e8-42b6-a8b9-e2041c3c0d24 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:38.128780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 828/1655] 10.222.0.65 () {54 vars in 912 bytes} [Tue May 10 20:16:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:16:38.128780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b8e28-9142-4bae-8bf7-24b122fa75e4 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:38.128780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b8e28-9142-4bae-8bf7-24b122fa75e4 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:38.128780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b8e28-9142-4bae-8bf7-24b122fa75e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:38.132700 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 828/1656] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:38.171279 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-199b8e28-9142-4bae-8bf7-24b122fa75e4 None None] MFA Rules not processed for user `ae922b2da4814d10bed1fa345700ac04`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:38.172543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-199b8e28-9142-4bae-8bf7-24b122fa75e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:38.323866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 829/1657] 10.222.0.65 () {58 vars in 1009 bytes} [Tue May 10 20:16:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:16:38.343235 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160be099-bbe2-41d7-abac-b4f0a951f246 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:38.344695 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-160be099-bbe2-41d7-abac-b4f0a951f246 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:38.421021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160be099-bbe2-41d7-abac-b4f0a951f246 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:38.430452 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33931bac-cc48-4828-952b-d69d4ca7cb41 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67a252cde71a455fb1dc4bf9c47c0490', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:38.431286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33931bac-cc48-4828-952b-d69d4ca7cb41 service glance] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:38.431802 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33931bac-cc48-4828-952b-d69d4ca7cb41 service glance] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:38.432121 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33931bac-cc48-4828-952b-d69d4ca7cb41 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:38.445578 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33931bac-cc48-4828-952b-d69d4ca7cb41 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:38.448349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33931bac-cc48-4828-952b-d69d4ca7cb41 service glance] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:38.482118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 829/1658] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:39.186465 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b8e28-9142-4bae-8bf7-24b122fa75e4 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:39.188004 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-199b8e28-9142-4bae-8bf7-24b122fa75e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:39.240819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b8e28-9142-4bae-8bf7-24b122fa75e4 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:39.265661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f7287-72b5-486a-a768-272de4185f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:39.267118 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f7287-72b5-486a-a768-272de4185f58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:39.267558 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f7287-72b5-486a-a768-272de4185f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:39.267997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f7287-72b5-486a-a768-272de4185f58 admin admin] PATH_INFO: `/v3/projects/2969433bca8d4ef494ac0cffe63f962d` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:39.271084 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1f7287-72b5-486a-a768-272de4185f58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2969433bca8d4ef494ac0cffe63f962d)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:39.287377 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1f7287-72b5-486a-a768-272de4185f58 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:39.325366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d4009-69f0-437b-8357-9f1782ed23be None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:39.326433 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d4009-69f0-437b-8357-9f1782ed23be None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:39.327061 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d4009-69f0-437b-8357-9f1782ed23be None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:39.331351 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 830/1659] 10.222.0.65 () {54 vars in 912 bytes} [Tue May 10 20:16:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 20:16:39.358197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d5a0f-6421-4d5e-b321-078a1b23bd2b None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:39.359295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d5a0f-6421-4d5e-b321-078a1b23bd2b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:39.359767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d5a0f-6421-4d5e-b321-078a1b23bd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:39.466344 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-952d5a0f-6421-4d5e-b321-078a1b23bd2b None None] MFA Rules not processed for user `a6aabd1571d04bcfae842626fef5ed6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:39.472026 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-952d5a0f-6421-4d5e-b321-078a1b23bd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:39.493011 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 831/1660] 10.222.0.65 () {58 vars in 1009 bytes} [Tue May 10 20:16:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:16:39.525760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d5a0f-6421-4d5e-b321-078a1b23bd2b None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:39.536629 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d5a0f-6421-4d5e-b321-078a1b23bd2b None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:39.540842 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3ee6a-bf19-4416-8a68-494a1542aeae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:39.541511 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3ee6a-bf19-4416-8a68-494a1542aeae service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:39.541846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3ee6a-bf19-4416-8a68-494a1542aeae service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:39.542159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3ee6a-bf19-4416-8a68-494a1542aeae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:39.552730 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3ee6a-bf19-4416-8a68-494a1542aeae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:39.554113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3ee6a-bf19-4416-8a68-494a1542aeae service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:39.573008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 832/1661] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:16:39.609334 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0b1f7287-72b5-486a-a768-272de4185f58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2969433bca8d4ef494ac0cffe63f962d', 'request_id': 'req-0b1f7287-72b5-486a-a768-272de4185f58'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:39.730268 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0b1f7287-72b5-486a-a768-272de4185f58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2969433bca8d4ef494ac0cffe63f962d', 'request_id': 'req-0b1f7287-72b5-486a-a768-272de4185f58'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:39.761435 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0b1f7287-72b5-486a-a768-272de4185f58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2969433bca8d4ef494ac0cffe63f962d', 'request_id': 'req-0b1f7287-72b5-486a-a768-272de4185f58'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:39.824369 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0b1f7287-72b5-486a-a768-272de4185f58 admin admin] The token cache is being invalidate because project 2969433bca8d4ef494ac0cffe63f962d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:16:39.824956 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0b1f7287-72b5-486a-a768-272de4185f58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:39.830556 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0b1f7287-72b5-486a-a768-272de4185f58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:39.834853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 830/1662] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:16:39 2022] DELETE /identity/v3/projects/2969433bca8d4ef494ac0cffe63f962d => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:39.838395 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3ee6a-bf19-4416-8a68-494a1542aeae service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:39.860226 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3ee6a-bf19-4416-8a68-494a1542aeae service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:40.030789 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9275fde-c7fd-48ef-bc7c-78372a1b28d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:40.032035 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9275fde-c7fd-48ef-bc7c-78372a1b28d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:40.032325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9275fde-c7fd-48ef-bc7c-78372a1b28d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:40.032936 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9275fde-c7fd-48ef-bc7c-78372a1b28d2 admin admin] PATH_INFO: `/v3/users/2612dcd40b8641f58cbeb186fc579974` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:40.034573 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9275fde-c7fd-48ef-bc7c-78372a1b28d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2612dcd40b8641f58cbeb186fc579974)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:40.037309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9275fde-c7fd-48ef-bc7c-78372a1b28d2 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:40.319590 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b9275fde-c7fd-48ef-bc7c-78372a1b28d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2612dcd40b8641f58cbeb186fc579974'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:40.320327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b9275fde-c7fd-48ef-bc7c-78372a1b28d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2612dcd40b8641f58cbeb186fc579974'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:40.321047 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b9275fde-c7fd-48ef-bc7c-78372a1b28d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2612dcd40b8641f58cbeb186fc579974'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:40.368777 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b9275fde-c7fd-48ef-bc7c-78372a1b28d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2612dcd40b8641f58cbeb186fc579974'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:40.385658 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b9275fde-c7fd-48ef-bc7c-78372a1b28d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2612dcd40b8641f58cbeb186fc579974'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:40.491273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 833/1663] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:16:39 2022] DELETE /identity/v3/users/2612dcd40b8641f58cbeb186fc579974 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:16:41.262521 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed70e6-fd99-478c-bdc8-02c1719cc0b7 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:41.263060 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed70e6-fd99-478c-bdc8-02c1719cc0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:41.263308 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed70e6-fd99-478c-bdc8-02c1719cc0b7 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:41.265282 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 831/1664] 10.222.0.65 () {54 vars in 960 bytes} [Tue May 10 20:16:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:16:41.272616 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c332cf-519e-4c5b-a2f0-802ddfa71865 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:41.273384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c332cf-519e-4c5b-a2f0-802ddfa71865 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:41.273799 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c332cf-519e-4c5b-a2f0-802ddfa71865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:41.316464 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-d3c332cf-519e-4c5b-a2f0-802ddfa71865 None None] MFA Rules not processed for user `022093dea10d46478b3016395c712d93`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:41.320667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-d3c332cf-519e-4c5b-a2f0-802ddfa71865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:41.540842 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 834/1665] 10.222.0.65 () {58 vars in 1057 bytes} [Tue May 10 20:16:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:41.541413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed70e6-fd99-478c-bdc8-02c1719cc0b7 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:41.574330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed70e6-fd99-478c-bdc8-02c1719cc0b7 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:41.581525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d03079-f6d0-43f3-a52f-648d99be9ce8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022093dea10d46478b3016395c712d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:41.588913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d03079-f6d0-43f3-a52f-648d99be9ce8 service placement] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:41.588913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d03079-f6d0-43f3-a52f-648d99be9ce8 service placement] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:41.588913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d03079-f6d0-43f3-a52f-648d99be9ce8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:41.602582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-23d03079-f6d0-43f3-a52f-648d99be9ce8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:41.632399 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d03079-f6d0-43f3-a52f-648d99be9ce8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:41.634107 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d03079-f6d0-43f3-a52f-648d99be9ce8 service placement] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:41.674565 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 832/1666] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:16:42.407230 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c332cf-519e-4c5b-a2f0-802ddfa71865 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:42.409597 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-d3c332cf-519e-4c5b-a2f0-802ddfa71865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:42.551911 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c332cf-519e-4c5b-a2f0-802ddfa71865 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:42.571743 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a2f77-9353-454c-a1b3-6beff2c7c059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:42.571743 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a2f77-9353-454c-a1b3-6beff2c7c059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:42.571743 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a2f77-9353-454c-a1b3-6beff2c7c059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:42.571743 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a2f77-9353-454c-a1b3-6beff2c7c059 admin admin] PATH_INFO: `/v3/projects/a023f221df7c45c1b166c8b63a0d7963` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:42.572009 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a2f77-9353-454c-a1b3-6beff2c7c059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a023f221df7c45c1b166c8b63a0d7963)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:42.599576 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: WARNING py.warnings [None req-5b1a2f77-9353-454c-a1b3-6beff2c7c059 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 20:16:42.599576 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: warnings.warn(msg) May 10 20:16:42.599576 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: May 10 20:16:42.600954 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a2f77-9353-454c-a1b3-6beff2c7c059 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:42.886575 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5b1a2f77-9353-454c-a1b3-6beff2c7c059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a023f221df7c45c1b166c8b63a0d7963', 'request_id': 'req-5b1a2f77-9353-454c-a1b3-6beff2c7c059'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:42.960439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5b1a2f77-9353-454c-a1b3-6beff2c7c059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a023f221df7c45c1b166c8b63a0d7963', 'request_id': 'req-5b1a2f77-9353-454c-a1b3-6beff2c7c059'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:43.007096 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5b1a2f77-9353-454c-a1b3-6beff2c7c059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a023f221df7c45c1b166c8b63a0d7963', 'request_id': 'req-5b1a2f77-9353-454c-a1b3-6beff2c7c059'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:43.065572 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5b1a2f77-9353-454c-a1b3-6beff2c7c059 admin admin] The token cache is being invalidate because project a023f221df7c45c1b166c8b63a0d7963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:16:43.065891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5b1a2f77-9353-454c-a1b3-6beff2c7c059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:43.090138 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5b1a2f77-9353-454c-a1b3-6beff2c7c059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:43.094195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 835/1667] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:16:42 2022] DELETE /identity/v3/projects/a023f221df7c45c1b166c8b63a0d7963 => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:43.097612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdafdf3-d6d6-4018-976a-d0b829172fad None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:43.099257 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdafdf3-d6d6-4018-976a-d0b829172fad None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:43.099700 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdafdf3-d6d6-4018-976a-d0b829172fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:43.140788 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-5bdafdf3-d6d6-4018-976a-d0b829172fad None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:43.142363 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-5bdafdf3-d6d6-4018-976a-d0b829172fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:43.268291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 833/1668] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:43.276607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b704370-acaf-4589-b19a-7d48c6aa4d41 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:43.276903 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b704370-acaf-4589-b19a-7d48c6aa4d41 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:43.277189 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b704370-acaf-4589-b19a-7d48c6aa4d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:43.334625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-3b704370-acaf-4589-b19a-7d48c6aa4d41 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:43.336197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3b704370-acaf-4589-b19a-7d48c6aa4d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:43.347587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 836/1669] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:43.352624 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdafdf3-d6d6-4018-976a-d0b829172fad None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:43.371029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdafdf3-d6d6-4018-976a-d0b829172fad None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:43.386136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a83221-112c-4a97-aa55-797749f4a711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:43.387473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a83221-112c-4a97-aa55-797749f4a711 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:43.387990 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a83221-112c-4a97-aa55-797749f4a711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:43.388464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a83221-112c-4a97-aa55-797749f4a711 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:43.389414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a83221-112c-4a97-aa55-797749f4a711 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:16:43.389925 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a83221-112c-4a97-aa55-797749f4a711 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:43.391838 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a83221-112c-4a97-aa55-797749f4a711 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:43.435782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 834/1670] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:16:43.442647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b704370-acaf-4589-b19a-7d48c6aa4d41 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:43.453361 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b704370-acaf-4589-b19a-7d48c6aa4d41 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:43.464203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81532597-671e-4630-9a93-b391fc4f7911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:43.464203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81532597-671e-4630-9a93-b391fc4f7911 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:43.464203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81532597-671e-4630-9a93-b391fc4f7911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:43.464203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81532597-671e-4630-9a93-b391fc4f7911 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:43.464715 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81532597-671e-4630-9a93-b391fc4f7911 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:43.470151 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81532597-671e-4630-9a93-b391fc4f7911 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:43.627944 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 837/1671] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:16:43 2022] POST /identity/v3/projects => generated 403 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:43.632437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a83221-112c-4a97-aa55-797749f4a711 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:43.649217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a83221-112c-4a97-aa55-797749f4a711 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:43.871440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f039c-69ee-455d-86cf-16a73e8b2108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:43.872372 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f039c-69ee-455d-86cf-16a73e8b2108 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:43.872798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f039c-69ee-455d-86cf-16a73e8b2108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:43.873978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f039c-69ee-455d-86cf-16a73e8b2108 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:43.875568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f039c-69ee-455d-86cf-16a73e8b2108 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:43.888552 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f039c-69ee-455d-86cf-16a73e8b2108 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:44.016926 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 835/1672] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:16:43 2022] POST /identity/v3/users => generated 302 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:44.017872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81532597-671e-4630-9a93-b391fc4f7911 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:44.034401 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81532597-671e-4630-9a93-b391fc4f7911 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:44.048483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e475fe-4be4-4945-a921-f9bea32e3660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:44.049744 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e475fe-4be4-4945-a921-f9bea32e3660 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:44.050226 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e475fe-4be4-4945-a921-f9bea32e3660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:44.050836 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e475fe-4be4-4945-a921-f9bea32e3660 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:44.052429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e475fe-4be4-4945-a921-f9bea32e3660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:44.056339 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e475fe-4be4-4945-a921-f9bea32e3660 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:44.083669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 838/1673] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:16:44.087938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f039c-69ee-455d-86cf-16a73e8b2108 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:44.100061 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f039c-69ee-455d-86cf-16a73e8b2108 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:44.115100 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c280c1-58ce-49c9-b626-8bc8798fa8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:44.117221 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c280c1-58ce-49c9-b626-8bc8798fa8ed admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:44.119584 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c280c1-58ce-49c9-b626-8bc8798fa8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:44.121812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c280c1-58ce-49c9-b626-8bc8798fa8ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:44.126976 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c280c1-58ce-49c9-b626-8bc8798fa8ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:44.134344 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c280c1-58ce-49c9-b626-8bc8798fa8ed admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:44.180491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 836/1674] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:44.185418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e475fe-4be4-4945-a921-f9bea32e3660 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:44.198423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e475fe-4be4-4945-a921-f9bea32e3660 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:44.203478 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4cfc8f-1a49-4816-bf11-4e166f89e9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:44.204543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cfc8f-1a49-4816-bf11-4e166f89e9c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:44.204996 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cfc8f-1a49-4816-bf11-4e166f89e9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:44.205416 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cfc8f-1a49-4816-bf11-4e166f89e9c8 admin admin] PATH_INFO: `/v3/projects/6334469fb3eb491099bd20fc59c07b11/users/dcb7aa38f91d432a94abc766e07bcee8/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:44.223969 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4cfc8f-1a49-4816-bf11-4e166f89e9c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6334469fb3eb491099bd20fc59c07b11, user_id=dcb7aa38f91d432a94abc766e07bcee8, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:44.227620 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4cfc8f-1a49-4816-bf11-4e166f89e9c8 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:44.362976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 839/1675] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:44 2022] PUT /identity/v3/projects/6334469fb3eb491099bd20fc59c07b11/users/dcb7aa38f91d432a94abc766e07bcee8/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:44.364236 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49a23a-3d64-4d17-97d2-2b6951cfd6a8 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:44.364697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49a23a-3d64-4d17-97d2-2b6951cfd6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:44.365169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49a23a-3d64-4d17-97d2-2b6951cfd6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:44.412823 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-3d49a23a-3d64-4d17-97d2-2b6951cfd6a8 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:44.416366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-3d49a23a-3d64-4d17-97d2-2b6951cfd6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:44.572498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 837/1676] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:44.635536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4cfc8f-1a49-4816-bf11-4e166f89e9c8 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:44.639595 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-0e4cfc8f-1a49-4816-bf11-4e166f89e9c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:44.861244 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4cfc8f-1a49-4816-bf11-4e166f89e9c8 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:44.883250 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc589d2b-d9e7-4e0c-b879-ec08d60ad40a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:44.883998 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc589d2b-d9e7-4e0c-b879-ec08d60ad40a service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:44.884287 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc589d2b-d9e7-4e0c-b879-ec08d60ad40a service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:44.884583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc589d2b-d9e7-4e0c-b879-ec08d60ad40a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:44.924580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc589d2b-d9e7-4e0c-b879-ec08d60ad40a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:44.926880 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc589d2b-d9e7-4e0c-b879-ec08d60ad40a service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:44.971260 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 840/1677] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:46.239284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1ba1b-2353-4be5-9561-b07a4ff78aa3 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:46.239792 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1ba1b-2353-4be5-9561-b07a4ff78aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:46.239909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1ba1b-2353-4be5-9561-b07a4ff78aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:46.311052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-06f1ba1b-2353-4be5-9561-b07a4ff78aa3 None None] MFA Rules not processed for user `261d01f5b22340a09708c71be64edae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:46.312335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-06f1ba1b-2353-4be5-9561-b07a4ff78aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:46.492941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 838/1678] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:46.506561 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73c771-a86f-4b1c-a7b7-a89d97c20ca8 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:46.508361 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73c771-a86f-4b1c-a7b7-a89d97c20ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:46.508781 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73c771-a86f-4b1c-a7b7-a89d97c20ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:46.543683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-9c73c771-a86f-4b1c-a7b7-a89d97c20ca8 None None] MFA Rules not processed for user `261d01f5b22340a09708c71be64edae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:46.545190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-9c73c771-a86f-4b1c-a7b7-a89d97c20ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:46.571125 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 841/1679] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:46.603268 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1ba1b-2353-4be5-9561-b07a4ff78aa3 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:46.605647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-06f1ba1b-2353-4be5-9561-b07a4ff78aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:46.865444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1ba1b-2353-4be5-9561-b07a4ff78aa3 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:46.872464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0387e-c1cd-4331-9f6f-57d76fe15dd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:46.873985 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0387e-c1cd-4331-9f6f-57d76fe15dd9 service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:46.874442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0387e-c1cd-4331-9f6f-57d76fe15dd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:46.874848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0387e-c1cd-4331-9f6f-57d76fe15dd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:46.885423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db0387e-c1cd-4331-9f6f-57d76fe15dd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:46.888426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db0387e-c1cd-4331-9f6f-57d76fe15dd9 service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:46.914462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 839/1680] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:16:47.211374 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b7552-8b63-4244-b139-a75efb1ba817 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:47.211888 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b7552-8b63-4244-b139-a75efb1ba817 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:47.218849 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b7552-8b63-4244-b139-a75efb1ba817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:47.234358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-6f4b7552-8b63-4244-b139-a75efb1ba817 None None] MFA Rules not processed for user `7ec2605fd1b84e0ea134b943ed33a605`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:47.235629 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-6f4b7552-8b63-4244-b139-a75efb1ba817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:47.338687 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 842/1681] 10.222.0.65 () {58 vars in 1009 bytes} [Tue May 10 20:16:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:47.389517 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0387e-c1cd-4331-9f6f-57d76fe15dd9 service nova] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:47.391899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8db0387e-c1cd-4331-9f6f-57d76fe15dd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:47.425303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0387e-c1cd-4331-9f6f-57d76fe15dd9 service nova] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:47.440233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228dd6d6-3fb0-44f2-bd29-40bf649afff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:47.441141 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228dd6d6-3fb0-44f2-bd29-40bf649afff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:47.441459 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228dd6d6-3fb0-44f2-bd29-40bf649afff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:47.441737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228dd6d6-3fb0-44f2-bd29-40bf649afff3 admin admin] PATH_INFO: `/v3/users/503075ec6bb947a6b21d009de442cec7` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:47.443217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228dd6d6-3fb0-44f2-bd29-40bf649afff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=503075ec6bb947a6b21d009de442cec7)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:47.444244 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: WARNING py.warnings [None req-228dd6d6-3fb0-44f2-bd29-40bf649afff3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 20:16:47.444244 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: warnings.warn(msg) May 10 20:16:47.444244 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: May 10 20:16:47.445448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228dd6d6-3fb0-44f2-bd29-40bf649afff3 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:47.687159 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-228dd6d6-3fb0-44f2-bd29-40bf649afff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '503075ec6bb947a6b21d009de442cec7'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:47.687851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-228dd6d6-3fb0-44f2-bd29-40bf649afff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '503075ec6bb947a6b21d009de442cec7'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:47.709397 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-228dd6d6-3fb0-44f2-bd29-40bf649afff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '503075ec6bb947a6b21d009de442cec7'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:47.710430 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-228dd6d6-3fb0-44f2-bd29-40bf649afff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '503075ec6bb947a6b21d009de442cec7'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:47.733454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-228dd6d6-3fb0-44f2-bd29-40bf649afff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '503075ec6bb947a6b21d009de442cec7'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:47.739787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b7552-8b63-4244-b139-a75efb1ba817 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:47.741511 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-6f4b7552-8b63-4244-b139-a75efb1ba817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:47.856758 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 840/1682] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:16:47 2022] DELETE /identity/v3/users/503075ec6bb947a6b21d009de442cec7 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:47.858281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ff818-8bb9-4273-a527-efce4bfc0227 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:47.858721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ff818-8bb9-4273-a527-efce4bfc0227 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:47.859172 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ff818-8bb9-4273-a527-efce4bfc0227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:47.933282 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-d49ff818-8bb9-4273-a527-efce4bfc0227 None None] MFA Rules not processed for user `dcb7aa38f91d432a94abc766e07bcee8`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:47.934687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-d49ff818-8bb9-4273-a527-efce4bfc0227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:48.089049 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b7552-8b63-4244-b139-a75efb1ba817 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:48.089625 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 841/1683] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:48.091191 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2b2e4-1e7c-42ed-a320-b02b8a24cfc9 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:48.093127 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2b2e4-1e7c-42ed-a320-b02b8a24cfc9 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:48.093969 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2b2e4-1e7c-42ed-a320-b02b8a24cfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:48.159020 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-e2c2b2e4-1e7c-42ed-a320-b02b8a24cfc9 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:48.162919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-e2c2b2e4-1e7c-42ed-a320-b02b8a24cfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:48.259483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efe156-b9b4-4d52-a823-953b508cacf2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d10b258a53484da22950e2b1db2e41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:48.261222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efe156-b9b4-4d52-a823-953b508cacf2 service swift] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:48.261521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efe156-b9b4-4d52-a823-953b508cacf2 service swift] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:48.261803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efe156-b9b4-4d52-a823-953b508cacf2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:48.268766 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-80efe156-b9b4-4d52-a823-953b508cacf2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:48.331522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 842/1684] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:48.332929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2b2e4-1e7c-42ed-a320-b02b8a24cfc9 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:48.338144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-e2c2b2e4-1e7c-42ed-a320-b02b8a24cfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:48.381761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2b2e4-1e7c-42ed-a320-b02b8a24cfc9 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:48.388735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80efe156-b9b4-4d52-a823-953b508cacf2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:48.390516 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80efe156-b9b4-4d52-a823-953b508cacf2 service swift] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:48.398458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccfa566-199c-4e29-962c-f39aaa2d30e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:48.399143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfa566-199c-4e29-962c-f39aaa2d30e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:48.399425 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfa566-199c-4e29-962c-f39aaa2d30e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:48.399702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfa566-199c-4e29-962c-f39aaa2d30e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:48.400446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccfa566-199c-4e29-962c-f39aaa2d30e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:48.408064 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccfa566-199c-4e29-962c-f39aaa2d30e8 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:48.443536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 843/1685] 10.222.0.65 () {60 vars in 1384 bytes} [Tue May 10 20:16:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 704 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 20:16:48.444572 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e2ec2-ea26-4a58-b713-655c15c4438b None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:48.444861 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e2ec2-ea26-4a58-b713-655c15c4438b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:48.445134 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e2ec2-ea26-4a58-b713-655c15c4438b None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:48.458991 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 844/1686] 10.222.0.65 () {54 vars in 912 bytes} [Tue May 10 20:16:48 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:16:48.459982 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e2ec2-ea26-4a58-b713-655c15c4438b None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:48.465312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-8b2e2ec2-ea26-4a58-b713-655c15c4438b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:48.586776 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 843/1687] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:16:48 2022] POST /identity/v3/projects => generated 403 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:48.594042 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7249da-e239-4a6e-9c7c-c7e8106acd7a None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:48.595471 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7249da-e239-4a6e-9c7c-c7e8106acd7a None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:48.604164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7249da-e239-4a6e-9c7c-c7e8106acd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:48.645456 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e2ec2-ea26-4a58-b713-655c15c4438b None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:48.703760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-fb7249da-e239-4a6e-9c7c-c7e8106acd7a None None] MFA Rules not processed for user `a6aabd1571d04bcfae842626fef5ed6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:48.705674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-fb7249da-e239-4a6e-9c7c-c7e8106acd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:48.784627 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c7a51-2d32-4e90-b898-f94e62d076b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:48.785372 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c7a51-2d32-4e90-b898-f94e62d076b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:48.785683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c7a51-2d32-4e90-b898-f94e62d076b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:48.785964 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c7a51-2d32-4e90-b898-f94e62d076b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:48.825447 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 844/1688] 10.222.0.65 () {58 vars in 1009 bytes} [Tue May 10 20:16:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:48.826765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7249da-e239-4a6e-9c7c-c7e8106acd7a None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:48.928731 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c7a51-2d32-4e90-b898-f94e62d076b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:48.930815 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c7a51-2d32-4e90-b898-f94e62d076b0 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:48.981425 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 845/1689] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:48.982588 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c7a51-2d32-4e90-b898-f94e62d076b0 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:48.990309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-396c7a51-2d32-4e90-b898-f94e62d076b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:49.000293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7249da-e239-4a6e-9c7c-c7e8106acd7a None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:49.028345 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c7a51-2d32-4e90-b898-f94e62d076b0 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:49.034243 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb67b38-bab0-4b3b-967b-4c9f8aeb9f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:49.035907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb67b38-bab0-4b3b-967b-4c9f8aeb9f13 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:49.036545 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb67b38-bab0-4b3b-967b-4c9f8aeb9f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:49.037022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb67b38-bab0-4b3b-967b-4c9f8aeb9f13 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:49.037989 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb67b38-bab0-4b3b-967b-4c9f8aeb9f13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:49.040348 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb67b38-bab0-4b3b-967b-4c9f8aeb9f13 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:49.042873 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1fcae-121a-433d-9e30-9de57a2658bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:49.044068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1fcae-121a-433d-9e30-9de57a2658bd service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:49.046824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1fcae-121a-433d-9e30-9de57a2658bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:49.048311 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1fcae-121a-433d-9e30-9de57a2658bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:49.082159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1fcae-121a-433d-9e30-9de57a2658bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:49.114115 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1fcae-121a-433d-9e30-9de57a2658bd service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:49.148317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 845/1690] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:16:48 2022] POST /identity/v3/users => generated 308 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:49.154976 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb67b38-bab0-4b3b-967b-4c9f8aeb9f13 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:49.176796 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 846/1691] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:49.189273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb67b38-bab0-4b3b-967b-4c9f8aeb9f13 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:49.214906 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985627c-4107-4137-9fe9-2d10480e5bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:49.215595 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985627c-4107-4137-9fe9-2d10480e5bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:49.215875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985627c-4107-4137-9fe9-2d10480e5bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:49.216183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985627c-4107-4137-9fe9-2d10480e5bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:49.217009 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985627c-4107-4137-9fe9-2d10480e5bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:49.218655 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985627c-4107-4137-9fe9-2d10480e5bd4 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:49.250837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1fcae-121a-433d-9e30-9de57a2658bd service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:49.252435 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 846/1692] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:16:49.273993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1fcae-121a-433d-9e30-9de57a2658bd service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:49.315935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32950c80-c557-4ba2-bd9e-b2e5abc29f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:49.316964 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32950c80-c557-4ba2-bd9e-b2e5abc29f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:49.317258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32950c80-c557-4ba2-bd9e-b2e5abc29f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:49.317515 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32950c80-c557-4ba2-bd9e-b2e5abc29f9a admin admin] PATH_INFO: `/v3/projects/8a21b78a1e5a438f828533f4f3d7024f/users/b14f14fa7a244b449f02ae7e24e035fb/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:49.362952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32950c80-c557-4ba2-bd9e-b2e5abc29f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a21b78a1e5a438f828533f4f3d7024f, user_id=b14f14fa7a244b449f02ae7e24e035fb, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:49.367140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32950c80-c557-4ba2-bd9e-b2e5abc29f9a admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:49.481683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 847/1693] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:49 2022] PUT /identity/v3/projects/8a21b78a1e5a438f828533f4f3d7024f/users/b14f14fa7a244b449f02ae7e24e035fb/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:49.482131 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985627c-4107-4137-9fe9-2d10480e5bd4 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:49.512793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985627c-4107-4137-9fe9-2d10480e5bd4 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:49.715674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52da183-0149-42ca-bdc4-c3c246614b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:49.716771 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52da183-0149-42ca-bdc4-c3c246614b41 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:49.717253 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52da183-0149-42ca-bdc4-c3c246614b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:49.717715 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52da183-0149-42ca-bdc4-c3c246614b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:49.718736 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52da183-0149-42ca-bdc4-c3c246614b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:49.728222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52da183-0149-42ca-bdc4-c3c246614b41 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:49.794608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 847/1694] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:16:49.795580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32950c80-c557-4ba2-bd9e-b2e5abc29f9a admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:49.861320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32950c80-c557-4ba2-bd9e-b2e5abc29f9a admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:49.886998 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526efb22-654f-490d-b902-3c2508df3759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:49.888524 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526efb22-654f-490d-b902-3c2508df3759 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:49.890782 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526efb22-654f-490d-b902-3c2508df3759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:49.891381 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526efb22-654f-490d-b902-3c2508df3759 admin admin] PATH_INFO: `/v3/projects/8a21b78a1e5a438f828533f4f3d7024f/users/b14f14fa7a244b449f02ae7e24e035fb/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:49.914897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526efb22-654f-490d-b902-3c2508df3759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a21b78a1e5a438f828533f4f3d7024f, user_id=b14f14fa7a244b449f02ae7e24e035fb, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:49.919385 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526efb22-654f-490d-b902-3c2508df3759 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:49.968485 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 848/1695] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:49 2022] PUT /identity/v3/projects/8a21b78a1e5a438f828533f4f3d7024f/users/b14f14fa7a244b449f02ae7e24e035fb/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:50.207106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52da183-0149-42ca-bdc4-c3c246614b41 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:50.209231 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a52da183-0149-42ca-bdc4-c3c246614b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:50.234921 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52da183-0149-42ca-bdc4-c3c246614b41 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:50.391977 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a624ca2-77ec-4569-a2d0-114d2cd3c083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:50.392861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a624ca2-77ec-4569-a2d0-114d2cd3c083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:50.393144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a624ca2-77ec-4569-a2d0-114d2cd3c083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:50.393699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a624ca2-77ec-4569-a2d0-114d2cd3c083 admin admin] PATH_INFO: `/v3/projects/051ca1834f2a4c2a84e885a473dab12b` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:50.395542 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a624ca2-77ec-4569-a2d0-114d2cd3c083 admin admin] RBAC: Authorizing `identity:delete_project(project_id=051ca1834f2a4c2a84e885a473dab12b)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:50.397601 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a624ca2-77ec-4569-a2d0-114d2cd3c083 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:50.666055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-5a624ca2-77ec-4569-a2d0-114d2cd3c083 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '051ca1834f2a4c2a84e885a473dab12b', 'request_id': 'req-5a624ca2-77ec-4569-a2d0-114d2cd3c083'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:50.763921 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-5a624ca2-77ec-4569-a2d0-114d2cd3c083 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '051ca1834f2a4c2a84e885a473dab12b', 'request_id': 'req-5a624ca2-77ec-4569-a2d0-114d2cd3c083'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:50.776478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-5a624ca2-77ec-4569-a2d0-114d2cd3c083 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '051ca1834f2a4c2a84e885a473dab12b', 'request_id': 'req-5a624ca2-77ec-4569-a2d0-114d2cd3c083'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:50.801622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-5a624ca2-77ec-4569-a2d0-114d2cd3c083 admin admin] The token cache is being invalidate because project 051ca1834f2a4c2a84e885a473dab12b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:16:50.802225 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-5a624ca2-77ec-4569-a2d0-114d2cd3c083 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:50.803610 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-5a624ca2-77ec-4569-a2d0-114d2cd3c083 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:16:50.807690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 848/1696] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:16:50 2022] DELETE /identity/v3/projects/051ca1834f2a4c2a84e885a473dab12b => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:50.812841 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730d38e-71e9-43ec-88b0-34359928bf8f None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:50.818775 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730d38e-71e9-43ec-88b0-34359928bf8f None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:50.833635 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730d38e-71e9-43ec-88b0-34359928bf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:50.908060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-4730d38e-71e9-43ec-88b0-34359928bf8f None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:50.909492 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4730d38e-71e9-43ec-88b0-34359928bf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:51.231591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 849/1697] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:51.240838 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e5663-6db6-4082-8009-0050922455fe None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:51.241292 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e5663-6db6-4082-8009-0050922455fe None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:51.241718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e5663-6db6-4082-8009-0050922455fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:51.324303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-1d6e5663-6db6-4082-8009-0050922455fe None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:51.329865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1d6e5663-6db6-4082-8009-0050922455fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:51.334186 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8ef89-0529-4382-a39d-44a67a5d8aca None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:51.334676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8ef89-0529-4382-a39d-44a67a5d8aca None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:51.335105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8ef89-0529-4382-a39d-44a67a5d8aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:51.380856 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 849/1698] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:51.389001 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e5663-6db6-4082-8009-0050922455fe None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:51.395780 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-bed8ef89-0529-4382-a39d-44a67a5d8aca None None] MFA Rules not processed for user `261d01f5b22340a09708c71be64edae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:51.397429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-bed8ef89-0529-4382-a39d-44a67a5d8aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:51.417233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e5663-6db6-4082-8009-0050922455fe None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:51.461818 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddc577-6c92-4e75-b90b-bb78d6ff45b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:51.462567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddc577-6c92-4e75-b90b-bb78d6ff45b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:51.462848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddc577-6c92-4e75-b90b-bb78d6ff45b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:51.463120 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddc577-6c92-4e75-b90b-bb78d6ff45b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:51.463869 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ddc577-6c92-4e75-b90b-bb78d6ff45b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:16:51.464174 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ddc577-6c92-4e75-b90b-bb78d6ff45b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:51.465958 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ddc577-6c92-4e75-b90b-bb78d6ff45b9 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:51.554725 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 850/1699] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:16:51.556005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddc577-6c92-4e75-b90b-bb78d6ff45b9 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:51.598672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddc577-6c92-4e75-b90b-bb78d6ff45b9 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:51.633208 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e9fd2-3da5-4d36-ad28-8bd994f95636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:51.634128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e9fd2-3da5-4d36-ad28-8bd994f95636 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:51.634254 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e9fd2-3da5-4d36-ad28-8bd994f95636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:51.634579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e9fd2-3da5-4d36-ad28-8bd994f95636 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:51.635413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e9fd2-3da5-4d36-ad28-8bd994f95636 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:51.637479 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e9fd2-3da5-4d36-ad28-8bd994f95636 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:51.648469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 850/1700] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:16:51.800207 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 851/1701] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:16:51 2022] POST /identity/v3/projects => generated 379 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:51.805045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8ef89-0529-4382-a39d-44a67a5d8aca None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:51.823304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8ef89-0529-4382-a39d-44a67a5d8aca None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:51.948173 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8df0c7-7c15-40ff-bd25-3dc11416823d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:51.953895 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8df0c7-7c15-40ff-bd25-3dc11416823d admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:51.953895 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8df0c7-7c15-40ff-bd25-3dc11416823d admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:51.953895 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8df0c7-7c15-40ff-bd25-3dc11416823d admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:51.953895 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8df0c7-7c15-40ff-bd25-3dc11416823d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:51.953895 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8df0c7-7c15-40ff-bd25-3dc11416823d admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:52.067232 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 851/1702] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:16:51 2022] POST /identity/v3/users => generated 290 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:16:52.071172 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e9fd2-3da5-4d36-ad28-8bd994f95636 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:52.104674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e9fd2-3da5-4d36-ad28-8bd994f95636 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:52.147027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb28e9-67e1-43a7-be77-9a6b8bcd2219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:52.147806 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb28e9-67e1-43a7-be77-9a6b8bcd2219 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:52.148137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb28e9-67e1-43a7-be77-9a6b8bcd2219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:52.148416 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb28e9-67e1-43a7-be77-9a6b8bcd2219 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:52.149199 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb28e9-67e1-43a7-be77-9a6b8bcd2219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:52.150923 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb28e9-67e1-43a7-be77-9a6b8bcd2219 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:52.229688 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8df0c7-7c15-40ff-bd25-3dc11416823d admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:52.245901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8df0c7-7c15-40ff-bd25-3dc11416823d admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:52.246688 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 852/1703] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:52.279857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd40064-57b2-420d-9661-4d187ddbf638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:52.284118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd40064-57b2-420d-9661-4d187ddbf638 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:52.284118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd40064-57b2-420d-9661-4d187ddbf638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:52.284118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd40064-57b2-420d-9661-4d187ddbf638 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:52.284118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd40064-57b2-420d-9661-4d187ddbf638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:52.284118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd40064-57b2-420d-9661-4d187ddbf638 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:52.300269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 852/1704] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:16:52.304419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb28e9-67e1-43a7-be77-9a6b8bcd2219 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:52.323852 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb28e9-67e1-43a7-be77-9a6b8bcd2219 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:52.362563 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45830204-75c3-43e3-81a5-b39727e5285f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:52.363612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45830204-75c3-43e3-81a5-b39727e5285f admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:52.363935 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45830204-75c3-43e3-81a5-b39727e5285f admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:52.364209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45830204-75c3-43e3-81a5-b39727e5285f admin admin] PATH_INFO: `/v3/projects/a520b50d4b47438b827bd2cfc5432270/users/9996d2c2b37e4a19a80211b79ee88a2d/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:52.431130 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45830204-75c3-43e3-81a5-b39727e5285f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a520b50d4b47438b827bd2cfc5432270, user_id=9996d2c2b37e4a19a80211b79ee88a2d, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:52.434627 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45830204-75c3-43e3-81a5-b39727e5285f admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:52.565292 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 853/1705] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:52 2022] PUT /identity/v3/projects/a520b50d4b47438b827bd2cfc5432270/users/9996d2c2b37e4a19a80211b79ee88a2d/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:52.576115 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f193f-8364-4017-82fd-304552bba065 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:52.576470 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f193f-8364-4017-82fd-304552bba065 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:52.576792 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f193f-8364-4017-82fd-304552bba065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:52.682280 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-a41f193f-8364-4017-82fd-304552bba065 None None] MFA Rules not processed for user `9996d2c2b37e4a19a80211b79ee88a2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:52.683946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a41f193f-8364-4017-82fd-304552bba065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:52.907964 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 853/1706] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:52.912112 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45830204-75c3-43e3-81a5-b39727e5285f admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:53.194492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45830204-75c3-43e3-81a5-b39727e5285f admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:53.200040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74b775-c726-4039-b7eb-62704b169cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:53.201434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74b775-c726-4039-b7eb-62704b169cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:53.205420 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74b775-c726-4039-b7eb-62704b169cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:53.205420 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74b775-c726-4039-b7eb-62704b169cc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:53.205420 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74b775-c726-4039-b7eb-62704b169cc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:53.205969 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74b775-c726-4039-b7eb-62704b169cc9 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:53.314551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f193f-8364-4017-82fd-304552bba065 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:53.314702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 854/1707] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:16:52 2022] POST /identity/v3/projects => generated 381 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:16:53.342701 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f193f-8364-4017-82fd-304552bba065 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:53.575247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c5516-4ad4-4ed5-873f-52dd47737ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:53.576307 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c5516-4ad4-4ed5-873f-52dd47737ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:53.576850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c5516-4ad4-4ed5-873f-52dd47737ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:53.577320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c5516-4ad4-4ed5-873f-52dd47737ef6 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:53.579482 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c5516-4ad4-4ed5-873f-52dd47737ef6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:53.604663 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c5516-4ad4-4ed5-873f-52dd47737ef6 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:53.718868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56178c8-6eb2-4f1e-9c51-47ec5bda49d3 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:53.719159 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56178c8-6eb2-4f1e-9c51-47ec5bda49d3 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:53.719446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56178c8-6eb2-4f1e-9c51-47ec5bda49d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:53.771184 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 854/1708] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:16:53 2022] POST /identity/v3/users => generated 297 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:16:53.788079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c5516-4ad4-4ed5-873f-52dd47737ef6 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:53.821985 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c5516-4ad4-4ed5-873f-52dd47737ef6 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:53.849658 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-a56178c8-6eb2-4f1e-9c51-47ec5bda49d3 None None] MFA Rules not processed for user `b14f14fa7a244b449f02ae7e24e035fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:53.856180 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a56178c8-6eb2-4f1e-9c51-47ec5bda49d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:53.872492 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4113220-c41a-4181-9609-eb4fd4e7a94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:53.874480 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4113220-c41a-4181-9609-eb4fd4e7a94b admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:53.874786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4113220-c41a-4181-9609-eb4fd4e7a94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:53.875065 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4113220-c41a-4181-9609-eb4fd4e7a94b admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:53.875820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4113220-c41a-4181-9609-eb4fd4e7a94b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:53.880537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4113220-c41a-4181-9609-eb4fd4e7a94b admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:53.928075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 855/1709] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:53.929665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4113220-c41a-4181-9609-eb4fd4e7a94b admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:53.963948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4113220-c41a-4181-9609-eb4fd4e7a94b admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:53.977118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0c31b-5b95-49a6-ab6e-7ddda75b3998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:53.986589 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0c31b-5b95-49a6-ab6e-7ddda75b3998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:53.986589 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0c31b-5b95-49a6-ab6e-7ddda75b3998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:53.986589 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0c31b-5b95-49a6-ab6e-7ddda75b3998 admin admin] PATH_INFO: `/v3/projects/b0ed9b7ec483461698d3a620216fa426/users/0bba3a68755b491c9589786d2ffaac3f/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:54.079281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0c31b-5b95-49a6-ab6e-7ddda75b3998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0ed9b7ec483461698d3a620216fa426, user_id=0bba3a68755b491c9589786d2ffaac3f, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:54.083574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 855/1710] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:53 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:16:54.085085 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0c31b-5b95-49a6-ab6e-7ddda75b3998 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:54.160095 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 856/1711] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:53 2022] PUT /identity/v3/projects/b0ed9b7ec483461698d3a620216fa426/users/0bba3a68755b491c9589786d2ffaac3f/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:54.164091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56178c8-6eb2-4f1e-9c51-47ec5bda49d3 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:54.179316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56178c8-6eb2-4f1e-9c51-47ec5bda49d3 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:54.301538 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85166dd-e842-43ec-ab00-4ee2128beb72 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:54.301835 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85166dd-e842-43ec-ab00-4ee2128beb72 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:54.302131 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85166dd-e842-43ec-ab00-4ee2128beb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:54.354187 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-a85166dd-e842-43ec-ab00-4ee2128beb72 None None] MFA Rules not processed for user `a6aabd1571d04bcfae842626fef5ed6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:54.356222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a85166dd-e842-43ec-ab00-4ee2128beb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:54.379880 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353575e-f697-47b0-953b-d84851802651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:54.382350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353575e-f697-47b0-953b-d84851802651 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:54.382879 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353575e-f697-47b0-953b-d84851802651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:54.383441 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353575e-f697-47b0-953b-d84851802651 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:54.385076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353575e-f697-47b0-953b-d84851802651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:54.397874 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353575e-f697-47b0-953b-d84851802651 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:54.431455 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 856/1712] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:16:54.435896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353575e-f697-47b0-953b-d84851802651 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:54.489097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353575e-f697-47b0-953b-d84851802651 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:54.502324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1cd045-292a-418f-8ead-728c62a60b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:54.505493 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cd045-292a-418f-8ead-728c62a60b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:54.506886 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cd045-292a-418f-8ead-728c62a60b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:54.508069 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cd045-292a-418f-8ead-728c62a60b50 admin admin] PATH_INFO: `/v3/projects/b0ed9b7ec483461698d3a620216fa426/users/0bba3a68755b491c9589786d2ffaac3f/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:54.514414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1cd045-292a-418f-8ead-728c62a60b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0ed9b7ec483461698d3a620216fa426, user_id=0bba3a68755b491c9589786d2ffaac3f, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:54.517622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1cd045-292a-418f-8ead-728c62a60b50 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:54.545325 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 857/1713] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:16:54 2022] PUT /identity/v3/projects/b0ed9b7ec483461698d3a620216fa426/users/0bba3a68755b491c9589786d2ffaac3f/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:16:54.555092 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09c1df-e21b-4148-88ef-3d127b3eff59 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:54.555768 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09c1df-e21b-4148-88ef-3d127b3eff59 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:54.556387 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09c1df-e21b-4148-88ef-3d127b3eff59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:54.595401 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 857/1714] 10.222.0.83 () {58 vars in 1015 bytes} [Tue May 10 20:16:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:16:54.626046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-1a09c1df-e21b-4148-88ef-3d127b3eff59 None None] MFA Rules not processed for user `0bba3a68755b491c9589786d2ffaac3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:54.627576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1a09c1df-e21b-4148-88ef-3d127b3eff59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:54.659657 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85166dd-e842-43ec-ab00-4ee2128beb72 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:54.663051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a85166dd-e842-43ec-ab00-4ee2128beb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:54.692406 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85166dd-e842-43ec-ab00-4ee2128beb72 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:54.877410 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 858/1715] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:54 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:54.881575 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef88930-ef0d-4bf3-ba98-b6ede4be7fb4 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:54.884839 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef88930-ef0d-4bf3-ba98-b6ede4be7fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:54.885948 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef88930-ef0d-4bf3-ba98-b6ede4be7fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:54.888457 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522364e-335f-45b0-bfb5-c0ba02f351df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:54.889314 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522364e-335f-45b0-bfb5-c0ba02f351df service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:54.890282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522364e-335f-45b0-bfb5-c0ba02f351df service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:54.891087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522364e-335f-45b0-bfb5-c0ba02f351df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:54.907833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522364e-335f-45b0-bfb5-c0ba02f351df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:54.913140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522364e-335f-45b0-bfb5-c0ba02f351df service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:54.964530 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 858/1716] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:54.967744 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6456d52-0b02-4f46-bcb1-9344d1cd5d95 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:54.968193 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6456d52-0b02-4f46-bcb1-9344d1cd5d95 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:54.968623 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6456d52-0b02-4f46-bcb1-9344d1cd5d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:54.986620 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-fef88930-ef0d-4bf3-ba98-b6ede4be7fb4 None None] MFA Rules not processed for user `35e69662bc7648b8b3e7467cf56c7f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:54.986620 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-fef88930-ef0d-4bf3-ba98-b6ede4be7fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:55.075164 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-a6456d52-0b02-4f46-bcb1-9344d1cd5d95 None None] MFA Rules not processed for user `8feebf7f09cb4f7d81759f8f1336dba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:16:55.084008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a6456d52-0b02-4f46-bcb1-9344d1cd5d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:55.252645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 859/1717] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:55.254917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef88930-ef0d-4bf3-ba98-b6ede4be7fb4 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:55.260046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-fef88930-ef0d-4bf3-ba98-b6ede4be7fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:55.412501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 859/1718] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:16:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:16:55.413437 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6456d52-0b02-4f46-bcb1-9344d1cd5d95 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:55.420552 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a6456d52-0b02-4f46-bcb1-9344d1cd5d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:55.586217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef88930-ef0d-4bf3-ba98-b6ede4be7fb4 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:55.629928 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01c352-ba21-4dc9-b585-763d309c3bb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:55.630498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01c352-ba21-4dc9-b585-763d309c3bb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:55.630783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01c352-ba21-4dc9-b585-763d309c3bb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:55.631104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01c352-ba21-4dc9-b585-763d309c3bb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:55.672411 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01c352-ba21-4dc9-b585-763d309c3bb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:55.678032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01c352-ba21-4dc9-b585-763d309c3bb9 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:55.720193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 860/1719] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:55.742868 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6456d52-0b02-4f46-bcb1-9344d1cd5d95 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:55.750796 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f5420-e66f-4ccf-ac8c-18fcfdecad0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:55.751508 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f5420-e66f-4ccf-ac8c-18fcfdecad0c service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:55.752256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f5420-e66f-4ccf-ac8c-18fcfdecad0c service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:55.752680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f5420-e66f-4ccf-ac8c-18fcfdecad0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:55.779289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f5420-e66f-4ccf-ac8c-18fcfdecad0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:55.780910 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f5420-e66f-4ccf-ac8c-18fcfdecad0c service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:55.806945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 860/1720] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:55.846299 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01c352-ba21-4dc9-b585-763d309c3bb9 service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:55.851242 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-ed01c352-ba21-4dc9-b585-763d309c3bb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:16:55.915724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01c352-ba21-4dc9-b585-763d309c3bb9 service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:55.933718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4174b829-7cda-4b38-a87a-a5cab981811f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:55.934739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174b829-7cda-4b38-a87a-a5cab981811f service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:55.935211 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174b829-7cda-4b38-a87a-a5cab981811f service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:55.935735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174b829-7cda-4b38-a87a-a5cab981811f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:55.958203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4174b829-7cda-4b38-a87a-a5cab981811f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:55.961565 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4174b829-7cda-4b38-a87a-a5cab981811f service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:55.996938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 861/1721] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:16:56.074662 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f5420-e66f-4ccf-ac8c-18fcfdecad0c service nova] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:16:56.086409 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f5420-e66f-4ccf-ac8c-18fcfdecad0c service nova] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:16:56.101493 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bc8e9-c505-4d0f-bd92-de1851a8f7c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:16:56.102591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bc8e9-c505-4d0f-bd92-de1851a8f7c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:16:56.103246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bc8e9-c505-4d0f-bd92-de1851a8f7c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:16:56.103816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bc8e9-c505-4d0f-bd92-de1851a8f7c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:16:56.259405 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bc8e9-c505-4d0f-bd92-de1851a8f7c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:16:56.261193 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bc8e9-c505-4d0f-bd92-de1851a8f7c2 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:16:56.279498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 861/1722] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:16:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:17:00.727113 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac9a39-8ea2-432d-bc47-573f80cfa120 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:00.728032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac9a39-8ea2-432d-bc47-573f80cfa120 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:00.728467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac9a39-8ea2-432d-bc47-573f80cfa120 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:00.730446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 862/1723] 10.222.0.65 () {54 vars in 921 bytes} [Tue May 10 20:17:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:17:00.762917 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a6538-aa50-4016-9ef9-72d324a1f77c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:00.764208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a6538-aa50-4016-9ef9-72d324a1f77c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:00.764805 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a6538-aa50-4016-9ef9-72d324a1f77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:00.801889 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-234a6538-aa50-4016-9ef9-72d324a1f77c None None] MFA Rules not processed for user `ae922b2da4814d10bed1fa345700ac04`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:00.803661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-234a6538-aa50-4016-9ef9-72d324a1f77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:00.834214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 862/1724] 10.222.0.65 () {58 vars in 1018 bytes} [Tue May 10 20:17:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:01.135935 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac9a39-8ea2-432d-bc47-573f80cfa120 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:01.175050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac9a39-8ea2-432d-bc47-573f80cfa120 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:01.182895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08c886-f575-466c-a88c-5fe8a58b8d3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:01.183769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08c886-f575-466c-a88c-5fe8a58b8d3a service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:01.184424 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08c886-f575-466c-a88c-5fe8a58b8d3a service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:01.185077 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08c886-f575-466c-a88c-5fe8a58b8d3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:01.201969 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08c886-f575-466c-a88c-5fe8a58b8d3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:01.205483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08c886-f575-466c-a88c-5fe8a58b8d3a service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:01.240544 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 863/1725] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:17:01.883180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a6538-aa50-4016-9ef9-72d324a1f77c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:01.884835 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-234a6538-aa50-4016-9ef9-72d324a1f77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:02.061203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a6538-aa50-4016-9ef9-72d324a1f77c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:02.066337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3424690-3cb7-4886-a060-234e95191a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:02.067707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3424690-3cb7-4886-a060-234e95191a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:02.068171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3424690-3cb7-4886-a060-234e95191a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:02.068729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3424690-3cb7-4886-a060-234e95191a91 admin admin] PATH_INFO: `/v3/users/dcb7aa38f91d432a94abc766e07bcee8` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:02.071080 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3424690-3cb7-4886-a060-234e95191a91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcb7aa38f91d432a94abc766e07bcee8)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:02.075949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3424690-3cb7-4886-a060-234e95191a91 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:02.343265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a3424690-3cb7-4886-a060-234e95191a91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcb7aa38f91d432a94abc766e07bcee8'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:02.344258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a3424690-3cb7-4886-a060-234e95191a91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcb7aa38f91d432a94abc766e07bcee8'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:02.345555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a3424690-3cb7-4886-a060-234e95191a91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcb7aa38f91d432a94abc766e07bcee8'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:02.363371 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a3424690-3cb7-4886-a060-234e95191a91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcb7aa38f91d432a94abc766e07bcee8'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:02.373232 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a3424690-3cb7-4886-a060-234e95191a91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcb7aa38f91d432a94abc766e07bcee8'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:02.470897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 863/1726] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:01 2022] DELETE /identity/v3/users/dcb7aa38f91d432a94abc766e07bcee8 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:02.934124 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08c886-f575-466c-a88c-5fe8a58b8d3a service nova] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:02.962704 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08c886-f575-466c-a88c-5fe8a58b8d3a service nova] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:03.124132 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94502d0a-3321-4501-9c04-8475f645d2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:03.125491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94502d0a-3321-4501-9c04-8475f645d2ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:03.127687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94502d0a-3321-4501-9c04-8475f645d2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:03.128158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94502d0a-3321-4501-9c04-8475f645d2ab admin admin] PATH_INFO: `/v3/projects/6334469fb3eb491099bd20fc59c07b11` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:03.130547 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94502d0a-3321-4501-9c04-8475f645d2ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=6334469fb3eb491099bd20fc59c07b11)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:03.132623 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94502d0a-3321-4501-9c04-8475f645d2ab admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:03.275980 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-94502d0a-3321-4501-9c04-8475f645d2ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6334469fb3eb491099bd20fc59c07b11', 'request_id': 'req-94502d0a-3321-4501-9c04-8475f645d2ab'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:03.365523 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-94502d0a-3321-4501-9c04-8475f645d2ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6334469fb3eb491099bd20fc59c07b11', 'request_id': 'req-94502d0a-3321-4501-9c04-8475f645d2ab'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:03.374434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-94502d0a-3321-4501-9c04-8475f645d2ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6334469fb3eb491099bd20fc59c07b11', 'request_id': 'req-94502d0a-3321-4501-9c04-8475f645d2ab'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:03.403707 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-94502d0a-3321-4501-9c04-8475f645d2ab admin admin] The token cache is being invalidate because project 6334469fb3eb491099bd20fc59c07b11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:03.403707 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-94502d0a-3321-4501-9c04-8475f645d2ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:03.403707 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-94502d0a-3321-4501-9c04-8475f645d2ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:03.403707 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 864/1727] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:02 2022] DELETE /identity/v3/projects/6334469fb3eb491099bd20fc59c07b11 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:03.407747 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3424690-3cb7-4886-a060-234e95191a91 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:03.410606 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a3424690-3cb7-4886-a060-234e95191a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:03.447612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3424690-3cb7-4886-a060-234e95191a91 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:03.542525 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a5da8-5ae2-47a3-8b86-78f9a35116e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:03.543770 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a5da8-5ae2-47a3-8b86-78f9a35116e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:03.544117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a5da8-5ae2-47a3-8b86-78f9a35116e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:03.544533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a5da8-5ae2-47a3-8b86-78f9a35116e7 admin admin] PATH_INFO: `/v3/users/b14f14fa7a244b449f02ae7e24e035fb` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:03.546214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a5da8-5ae2-47a3-8b86-78f9a35116e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b14f14fa7a244b449f02ae7e24e035fb)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:03.549091 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a5da8-5ae2-47a3-8b86-78f9a35116e7 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:03.779935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-528a5da8-5ae2-47a3-8b86-78f9a35116e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b14f14fa7a244b449f02ae7e24e035fb'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:03.780624 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-528a5da8-5ae2-47a3-8b86-78f9a35116e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b14f14fa7a244b449f02ae7e24e035fb'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:03.781223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-528a5da8-5ae2-47a3-8b86-78f9a35116e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b14f14fa7a244b449f02ae7e24e035fb'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:03.795242 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-528a5da8-5ae2-47a3-8b86-78f9a35116e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b14f14fa7a244b449f02ae7e24e035fb'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:03.805862 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-528a5da8-5ae2-47a3-8b86-78f9a35116e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b14f14fa7a244b449f02ae7e24e035fb'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:03.868654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 864/1728] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:03 2022] DELETE /identity/v3/users/b14f14fa7a244b449f02ae7e24e035fb => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:04.130899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94502d0a-3321-4501-9c04-8475f645d2ab admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:04.132750 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-94502d0a-3321-4501-9c04-8475f645d2ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:04.154775 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94502d0a-3321-4501-9c04-8475f645d2ab admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:04.209917 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a1c0c-97ab-4040-97fd-8dbe68e2da4d None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:04.212769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a1c0c-97ab-4040-97fd-8dbe68e2da4d None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:04.212769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a1c0c-97ab-4040-97fd-8dbe68e2da4d None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:04.212769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 865/1729] 10.222.0.65 () {54 vars in 921 bytes} [Tue May 10 20:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 20:17:04.233715 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3994d07-0253-4085-9a71-f3ef7a7ebb9e None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:04.233983 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3994d07-0253-4085-9a71-f3ef7a7ebb9e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:04.234300 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3994d07-0253-4085-9a71-f3ef7a7ebb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:04.255298 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fbe2ea-625b-4d5d-b673-2eedfa3f5e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:04.256735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbe2ea-625b-4d5d-b673-2eedfa3f5e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:04.257176 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbe2ea-625b-4d5d-b673-2eedfa3f5e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:04.257604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbe2ea-625b-4d5d-b673-2eedfa3f5e0e admin admin] PATH_INFO: `/v3/projects/8a21b78a1e5a438f828533f4f3d7024f` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:04.259728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fbe2ea-625b-4d5d-b673-2eedfa3f5e0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a21b78a1e5a438f828533f4f3d7024f)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:04.262070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fbe2ea-625b-4d5d-b673-2eedfa3f5e0e admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:04.271094 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-e3994d07-0253-4085-9a71-f3ef7a7ebb9e None None] MFA Rules not processed for user `ae922b2da4814d10bed1fa345700ac04`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:04.272552 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-e3994d07-0253-4085-9a71-f3ef7a7ebb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:04.423212 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-25fbe2ea-625b-4d5d-b673-2eedfa3f5e0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a21b78a1e5a438f828533f4f3d7024f', 'request_id': 'req-25fbe2ea-625b-4d5d-b673-2eedfa3f5e0e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:04.441461 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 866/1730] 10.222.0.65 () {58 vars in 1018 bytes} [Tue May 10 20:17:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:04.462212 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b379f4a-38c9-4458-a8ef-3e5e5712c66d None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:04.466587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b379f4a-38c9-4458-a8ef-3e5e5712c66d None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:04.466931 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b379f4a-38c9-4458-a8ef-3e5e5712c66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:04.501678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-1b379f4a-38c9-4458-a8ef-3e5e5712c66d None None] MFA Rules not processed for user `35e69662bc7648b8b3e7467cf56c7f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:04.503173 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-1b379f4a-38c9-4458-a8ef-3e5e5712c66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:04.507504 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-25fbe2ea-625b-4d5d-b673-2eedfa3f5e0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a21b78a1e5a438f828533f4f3d7024f', 'request_id': 'req-25fbe2ea-625b-4d5d-b673-2eedfa3f5e0e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:04.520268 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-25fbe2ea-625b-4d5d-b673-2eedfa3f5e0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a21b78a1e5a438f828533f4f3d7024f', 'request_id': 'req-25fbe2ea-625b-4d5d-b673-2eedfa3f5e0e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:04.557009 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-25fbe2ea-625b-4d5d-b673-2eedfa3f5e0e admin admin] The token cache is being invalidate because project 8a21b78a1e5a438f828533f4f3d7024f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:04.557325 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-25fbe2ea-625b-4d5d-b673-2eedfa3f5e0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:04.557997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-25fbe2ea-625b-4d5d-b673-2eedfa3f5e0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:04.574143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 865/1731] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:04 2022] DELETE /identity/v3/projects/8a21b78a1e5a438f828533f4f3d7024f => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:04.575341 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fbe2ea-625b-4d5d-b673-2eedfa3f5e0e admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:04.579375 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-25fbe2ea-625b-4d5d-b673-2eedfa3f5e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:04.624313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 867/1732] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:04.626016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c1161-00d2-4ece-a408-16572e589905 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:04.626606 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c1161-00d2-4ece-a408-16572e589905 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:04.627109 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c1161-00d2-4ece-a408-16572e589905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:04.654110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-7f2c1161-00d2-4ece-a408-16572e589905 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:04.656012 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7f2c1161-00d2-4ece-a408-16572e589905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:04.798278 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fbe2ea-625b-4d5d-b673-2eedfa3f5e0e admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:04.819939 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 868/1733] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:04.821462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901e282-c8be-44b0-b76f-af0397521bdf None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:04.821729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901e282-c8be-44b0-b76f-af0397521bdf None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:04.822010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901e282-c8be-44b0-b76f-af0397521bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:04.832798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78261a8-0f13-40ac-8126-a4775e5f621b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:04.833531 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78261a8-0f13-40ac-8126-a4775e5f621b service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:04.833870 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78261a8-0f13-40ac-8126-a4775e5f621b service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:04.836827 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78261a8-0f13-40ac-8126-a4775e5f621b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:04.843420 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a78261a8-0f13-40ac-8126-a4775e5f621b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:04.873311 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-3901e282-c8be-44b0-b76f-af0397521bdf None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:04.875438 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78261a8-0f13-40ac-8126-a4775e5f621b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:04.878440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3901e282-c8be-44b0-b76f-af0397521bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:04.881635 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78261a8-0f13-40ac-8126-a4775e5f621b service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:04.902308 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 869/1734] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:04.904974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 866/1735] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:17:04.906671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901e282-c8be-44b0-b76f-af0397521bdf None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:04.920988 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901e282-c8be-44b0-b76f-af0397521bdf None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:04.929021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb2e1e-9edd-41d9-9b7e-7a05b36fb7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:04.930786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb2e1e-9edd-41d9-9b7e-7a05b36fb7b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:04.931212 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb2e1e-9edd-41d9-9b7e-7a05b36fb7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:04.931574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb2e1e-9edd-41d9-9b7e-7a05b36fb7b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:04.933294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb2e1e-9edd-41d9-9b7e-7a05b36fb7b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:17:04.933800 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb2e1e-9edd-41d9-9b7e-7a05b36fb7b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:04.935824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb2e1e-9edd-41d9-9b7e-7a05b36fb7b2 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:04.955452 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 870/1736] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:17:04.957137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78261a8-0f13-40ac-8126-a4775e5f621b service nova] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:04.971953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78261a8-0f13-40ac-8126-a4775e5f621b service nova] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:04.974713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db30fd3-1f05-46eb-b31d-b334e9f22485 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:04.975011 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db30fd3-1f05-46eb-b31d-b334e9f22485 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:04.975296 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db30fd3-1f05-46eb-b31d-b334e9f22485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:04.993925 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b891c-9508-45b1-8f56-8b3ee4a11cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:04.994632 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b891c-9508-45b1-8f56-8b3ee4a11cac admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:04.994920 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b891c-9508-45b1-8f56-8b3ee4a11cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:04.995195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b891c-9508-45b1-8f56-8b3ee4a11cac admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:04.995927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b891c-9508-45b1-8f56-8b3ee4a11cac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:05.007660 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b891c-9508-45b1-8f56-8b3ee4a11cac admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:05.039092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-9db30fd3-1f05-46eb-b31d-b334e9f22485 None None] MFA Rules not processed for user `8feebf7f09cb4f7d81759f8f1336dba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:05.046257 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-9db30fd3-1f05-46eb-b31d-b334e9f22485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:05.153067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 867/1737] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:04 2022] POST /identity/v3/projects => generated 367 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:05.159113 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b891c-9508-45b1-8f56-8b3ee4a11cac admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:05.175244 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 871/1738] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:05.339292 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b891c-9508-45b1-8f56-8b3ee4a11cac admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:05.355980 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8c49e-0a16-48f6-8d5d-8dda1181ff12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:05.357004 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8c49e-0a16-48f6-8d5d-8dda1181ff12 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:05.357439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8c49e-0a16-48f6-8d5d-8dda1181ff12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:05.357889 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8c49e-0a16-48f6-8d5d-8dda1181ff12 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:05.358796 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee8c49e-0a16-48f6-8d5d-8dda1181ff12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:05.361432 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee8c49e-0a16-48f6-8d5d-8dda1181ff12 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:05.486412 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db30fd3-1f05-46eb-b31d-b334e9f22485 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:05.488725 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-9db30fd3-1f05-46eb-b31d-b334e9f22485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:05.498451 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 868/1739] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:05 2022] POST /identity/v3/users => generated 284 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:17:05.504529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8c49e-0a16-48f6-8d5d-8dda1181ff12 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:05.520092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db30fd3-1f05-46eb-b31d-b334e9f22485 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:05.523243 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8c49e-0a16-48f6-8d5d-8dda1181ff12 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:05.527466 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da17baa-8d56-4f07-8680-13dc2a254e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:05.528914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da17baa-8d56-4f07-8680-13dc2a254e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:05.530320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da17baa-8d56-4f07-8680-13dc2a254e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:05.531054 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da17baa-8d56-4f07-8680-13dc2a254e69 admin admin] PATH_INFO: `/v3/users/56be7d9ead0f4e1f8d12c29e62e40e1f` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:05.533238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da17baa-8d56-4f07-8680-13dc2a254e69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56be7d9ead0f4e1f8d12c29e62e40e1f)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:05.534693 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78157a3-695a-4eaf-8afd-ce5d267f8f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:05.535407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78157a3-695a-4eaf-8afd-ce5d267f8f7d admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:05.535701 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78157a3-695a-4eaf-8afd-ce5d267f8f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:05.535960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78157a3-695a-4eaf-8afd-ce5d267f8f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:05.536733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78157a3-695a-4eaf-8afd-ce5d267f8f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:05.541035 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78157a3-695a-4eaf-8afd-ce5d267f8f7d admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:05.549335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da17baa-8d56-4f07-8680-13dc2a254e69 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:05.573681 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 869/1740] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:05.575197 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78157a3-695a-4eaf-8afd-ce5d267f8f7d admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:05.595948 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78157a3-695a-4eaf-8afd-ce5d267f8f7d admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:05.610189 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfce10-4683-433e-ad48-19eb8e815712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:05.610856 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfce10-4683-433e-ad48-19eb8e815712 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:05.611209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfce10-4683-433e-ad48-19eb8e815712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:05.611595 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfce10-4683-433e-ad48-19eb8e815712 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:05.612353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bfce10-4683-433e-ad48-19eb8e815712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:05.618891 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bfce10-4683-433e-ad48-19eb8e815712 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:05.662610 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 870/1741] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:17:05.663746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfce10-4683-433e-ad48-19eb8e815712 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:05.705532 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfce10-4683-433e-ad48-19eb8e815712 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:05.717088 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fdc2f-5921-4d7d-837c-37f4f25646da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:05.719554 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fdc2f-5921-4d7d-837c-37f4f25646da admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:05.721191 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fdc2f-5921-4d7d-837c-37f4f25646da admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:05.722694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fdc2f-5921-4d7d-837c-37f4f25646da admin admin] PATH_INFO: `/v3/projects/1843d1fa62bd4ee2b05c46b7000f36b2/users/fa7fbd21a8a447adb2775a24e9894541/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:05.767997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fdc2f-5921-4d7d-837c-37f4f25646da admin admin] RBAC: Authorizing `identity:create_grant(project_id=1843d1fa62bd4ee2b05c46b7000f36b2, user_id=fa7fbd21a8a447adb2775a24e9894541, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:05.773842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fdc2f-5921-4d7d-837c-37f4f25646da admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:05.817247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6da17baa-8d56-4f07-8680-13dc2a254e69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56be7d9ead0f4e1f8d12c29e62e40e1f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:05.818041 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6da17baa-8d56-4f07-8680-13dc2a254e69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56be7d9ead0f4e1f8d12c29e62e40e1f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:05.819098 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6da17baa-8d56-4f07-8680-13dc2a254e69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56be7d9ead0f4e1f8d12c29e62e40e1f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:05.845752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6da17baa-8d56-4f07-8680-13dc2a254e69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56be7d9ead0f4e1f8d12c29e62e40e1f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:05.863871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6da17baa-8d56-4f07-8680-13dc2a254e69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56be7d9ead0f4e1f8d12c29e62e40e1f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:05.901243 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 871/1742] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:05 2022] PUT /identity/v3/projects/1843d1fa62bd4ee2b05c46b7000f36b2/users/fa7fbd21a8a447adb2775a24e9894541/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:05.912527 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3202fe-4821-4597-b43a-1a80e8760441 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:05.913111 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3202fe-4821-4597-b43a-1a80e8760441 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:05.913668 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3202fe-4821-4597-b43a-1a80e8760441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:05.946286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-2a3202fe-4821-4597-b43a-1a80e8760441 None None] MFA Rules not processed for user `fa7fbd21a8a447adb2775a24e9894541`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:05.947588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-2a3202fe-4821-4597-b43a-1a80e8760441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:05.976534 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 872/1743] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:05 2022] DELETE /identity/v3/users/56be7d9ead0f4e1f8d12c29e62e40e1f => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:05.980776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b9abe-f7c2-4000-ae3a-c98e0d62bfa8 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:05.981286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b9abe-f7c2-4000-ae3a-c98e0d62bfa8 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:05.981734 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b9abe-f7c2-4000-ae3a-c98e0d62bfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:06.023522 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-232b9abe-f7c2-4000-ae3a-c98e0d62bfa8 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:06.024977 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-232b9abe-f7c2-4000-ae3a-c98e0d62bfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:06.142923 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 872/1744] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:05 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:06.149339 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3202fe-4821-4597-b43a-1a80e8760441 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:06.152286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-2a3202fe-4821-4597-b43a-1a80e8760441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:06.156648 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 873/1745] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:06.230016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b9abe-f7c2-4000-ae3a-c98e0d62bfa8 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:06.232233 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-232b9abe-f7c2-4000-ae3a-c98e0d62bfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:06.350725 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3202fe-4821-4597-b43a-1a80e8760441 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:06.354418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b9abe-f7c2-4000-ae3a-c98e0d62bfa8 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:06.357204 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785e241-fda1-4837-814f-d1d2c88b706d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:06.360799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785e241-fda1-4837-814f-d1d2c88b706d admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:06.361318 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785e241-fda1-4837-814f-d1d2c88b706d admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:06.361794 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785e241-fda1-4837-814f-d1d2c88b706d admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:06.362735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8785e241-fda1-4837-814f-d1d2c88b706d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:06.365179 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8785e241-fda1-4837-814f-d1d2c88b706d admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:06.373826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e7f79-1f61-429a-8191-709d7d947e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:06.373826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e7f79-1f61-429a-8191-709d7d947e6e service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:06.373826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e7f79-1f61-429a-8191-709d7d947e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:06.373826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e7f79-1f61-429a-8191-709d7d947e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:06.388573 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e7f79-1f61-429a-8191-709d7d947e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:06.390037 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e7f79-1f61-429a-8191-709d7d947e6e service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:06.406918 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 874/1746] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:17:06.476036 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 873/1747] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:06 2022] POST /identity/v3/projects => generated 365 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:06.480493 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e7f79-1f61-429a-8191-709d7d947e6e service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:06.495359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e7f79-1f61-429a-8191-709d7d947e6e service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:06.563414 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8fae4-150e-44b7-b633-56f027c94ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:06.564222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8fae4-150e-44b7-b633-56f027c94ede admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:06.564572 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8fae4-150e-44b7-b633-56f027c94ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:06.564963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8fae4-150e-44b7-b633-56f027c94ede admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:06.565748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8fae4-150e-44b7-b633-56f027c94ede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:06.568679 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8fae4-150e-44b7-b633-56f027c94ede admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:06.653321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 875/1748] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:06 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:06.654329 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785e241-fda1-4837-814f-d1d2c88b706d admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:06.665175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785e241-fda1-4837-814f-d1d2c88b706d admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:06.670475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a974968-04e2-4789-85d4-868561b66c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:06.682827 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a974968-04e2-4789-85d4-868561b66c5a admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:06.682827 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a974968-04e2-4789-85d4-868561b66c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:06.682827 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a974968-04e2-4789-85d4-868561b66c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:06.682827 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a974968-04e2-4789-85d4-868561b66c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:06.682827 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a974968-04e2-4789-85d4-868561b66c5a admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:06.689950 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 874/1749] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:17:06.693785 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8fae4-150e-44b7-b633-56f027c94ede admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:06.711500 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8fae4-150e-44b7-b633-56f027c94ede admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:06.718974 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78195727-3c9b-402c-8cb2-49c74b567247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:06.720266 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78195727-3c9b-402c-8cb2-49c74b567247 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:06.721692 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78195727-3c9b-402c-8cb2-49c74b567247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:06.722520 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78195727-3c9b-402c-8cb2-49c74b567247 admin admin] PATH_INFO: `/v3/projects/e83ad63d060e4abe95356d1efb54ad17/users/8f3c2acd8c9049299186275db8e727ac/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:06.747052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78195727-3c9b-402c-8cb2-49c74b567247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e83ad63d060e4abe95356d1efb54ad17, user_id=8f3c2acd8c9049299186275db8e727ac, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:06.750098 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78195727-3c9b-402c-8cb2-49c74b567247 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:06.887932 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 876/1750] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:06 2022] PUT /identity/v3/projects/e83ad63d060e4abe95356d1efb54ad17/users/8f3c2acd8c9049299186275db8e727ac/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:06.893296 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a974968-04e2-4789-85d4-868561b66c5a admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:06.917676 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a974968-04e2-4789-85d4-868561b66c5a admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:07.030807 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eeefc8-4f02-47ec-99c1-8880f610ee2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:07.031795 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eeefc8-4f02-47ec-99c1-8880f610ee2f admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:07.032232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eeefc8-4f02-47ec-99c1-8880f610ee2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:07.032659 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eeefc8-4f02-47ec-99c1-8880f610ee2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:07.033549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eeefc8-4f02-47ec-99c1-8880f610ee2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:07.038540 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eeefc8-4f02-47ec-99c1-8880f610ee2f admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:07.053432 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 875/1751] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:07.057485 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78195727-3c9b-402c-8cb2-49c74b567247 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:07.072256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78195727-3c9b-402c-8cb2-49c74b567247 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:07.079350 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae28898-e171-4ec8-9db6-8cf90f6f3a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:07.080266 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae28898-e171-4ec8-9db6-8cf90f6f3a8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:07.080566 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae28898-e171-4ec8-9db6-8cf90f6f3a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:07.080942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae28898-e171-4ec8-9db6-8cf90f6f3a8e admin admin] PATH_INFO: `/v3/projects/e83ad63d060e4abe95356d1efb54ad17/users/8f3c2acd8c9049299186275db8e727ac/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:07.083045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae28898-e171-4ec8-9db6-8cf90f6f3a8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e83ad63d060e4abe95356d1efb54ad17, user_id=8f3c2acd8c9049299186275db8e727ac, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:07.086222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae28898-e171-4ec8-9db6-8cf90f6f3a8e admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:07.115203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 877/1752] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:07 2022] PUT /identity/v3/projects/e83ad63d060e4abe95356d1efb54ad17/users/8f3c2acd8c9049299186275db8e727ac/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:07.126204 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff7aa1-b470-42c3-92f6-24750acf21ba None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:07.126502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff7aa1-b470-42c3-92f6-24750acf21ba None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:07.126785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff7aa1-b470-42c3-92f6-24750acf21ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:07.174665 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-08ff7aa1-b470-42c3-92f6-24750acf21ba None None] MFA Rules not processed for user `8f3c2acd8c9049299186275db8e727ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:07.175921 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-08ff7aa1-b470-42c3-92f6-24750acf21ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:07.305829 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 876/1753] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:07 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:07.317779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae28898-e171-4ec8-9db6-8cf90f6f3a8e admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:07.320007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-eae28898-e171-4ec8-9db6-8cf90f6f3a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:07.474345 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae28898-e171-4ec8-9db6-8cf90f6f3a8e admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:07.484049 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f605ff6-0489-442b-a9ef-5cce9c928775 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:07.485450 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f605ff6-0489-442b-a9ef-5cce9c928775 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:07.486130 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f605ff6-0489-442b-a9ef-5cce9c928775 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:07.486583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f605ff6-0489-442b-a9ef-5cce9c928775 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:07.501002 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f605ff6-0489-442b-a9ef-5cce9c928775 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:07.504327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f605ff6-0489-442b-a9ef-5cce9c928775 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:07.518153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 878/1754] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:07 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:17:07.538337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff7aa1-b470-42c3-92f6-24750acf21ba None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:07.540503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-08ff7aa1-b470-42c3-92f6-24750acf21ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:07.739016 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff7aa1-b470-42c3-92f6-24750acf21ba None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:07.746414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08592e83-a12d-47ee-aebe-9823f1c9b8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:07.747360 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08592e83-a12d-47ee-aebe-9823f1c9b8d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:07.747643 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08592e83-a12d-47ee-aebe-9823f1c9b8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:07.747914 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08592e83-a12d-47ee-aebe-9823f1c9b8d0 admin admin] PATH_INFO: `/v3/projects/d3967d950ee74522aeee6b82e4b4615a` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:07.749601 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08592e83-a12d-47ee-aebe-9823f1c9b8d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3967d950ee74522aeee6b82e4b4615a)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:07.752147 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08592e83-a12d-47ee-aebe-9823f1c9b8d0 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:07.909920 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-08592e83-a12d-47ee-aebe-9823f1c9b8d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3967d950ee74522aeee6b82e4b4615a', 'request_id': 'req-08592e83-a12d-47ee-aebe-9823f1c9b8d0'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:08.038916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-08592e83-a12d-47ee-aebe-9823f1c9b8d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3967d950ee74522aeee6b82e4b4615a', 'request_id': 'req-08592e83-a12d-47ee-aebe-9823f1c9b8d0'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:08.045999 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-08592e83-a12d-47ee-aebe-9823f1c9b8d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3967d950ee74522aeee6b82e4b4615a', 'request_id': 'req-08592e83-a12d-47ee-aebe-9823f1c9b8d0'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:08.058882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f605ff6-0489-442b-a9ef-5cce9c928775 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:08.069179 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-08592e83-a12d-47ee-aebe-9823f1c9b8d0 admin admin] The token cache is being invalidate because project d3967d950ee74522aeee6b82e4b4615a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:08.069467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-08592e83-a12d-47ee-aebe-9823f1c9b8d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:08.070202 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-08592e83-a12d-47ee-aebe-9823f1c9b8d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:08.070320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f605ff6-0489-442b-a9ef-5cce9c928775 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:08.071991 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 877/1755] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:07 2022] DELETE /identity/v3/projects/d3967d950ee74522aeee6b82e4b4615a => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:08.075463 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08592e83-a12d-47ee-aebe-9823f1c9b8d0 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:08.077127 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-08592e83-a12d-47ee-aebe-9823f1c9b8d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:08.121475 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf062a-7d2a-424f-9f25-f03dfb3cd1ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:08.122246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf062a-7d2a-424f-9f25-f03dfb3cd1ca service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:08.122606 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf062a-7d2a-424f-9f25-f03dfb3cd1ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:08.123040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf062a-7d2a-424f-9f25-f03dfb3cd1ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:08.125141 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-acaf062a-7d2a-424f-9f25-f03dfb3cd1ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:08.145366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08592e83-a12d-47ee-aebe-9823f1c9b8d0 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:08.150223 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6510c-ffe6-4c83-bad9-635aeba91586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:08.152324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6510c-ffe6-4c83-bad9-635aeba91586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:08.152324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6510c-ffe6-4c83-bad9-635aeba91586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:08.152324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6510c-ffe6-4c83-bad9-635aeba91586 admin admin] PATH_INFO: `/v3/users/2a72b372c8d0469e815fbe536b4cf9bb` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:08.152830 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6510c-ffe6-4c83-bad9-635aeba91586 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a72b372c8d0469e815fbe536b4cf9bb)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:08.154857 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6510c-ffe6-4c83-bad9-635aeba91586 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:08.193274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf062a-7d2a-424f-9f25-f03dfb3cd1ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:08.194832 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf062a-7d2a-424f-9f25-f03dfb3cd1ca service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:08.204025 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 879/1756] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:08 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:17:08.345457 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-65b6510c-ffe6-4c83-bad9-635aeba91586 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a72b372c8d0469e815fbe536b4cf9bb'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:08.346995 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-65b6510c-ffe6-4c83-bad9-635aeba91586 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a72b372c8d0469e815fbe536b4cf9bb'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:08.361864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-65b6510c-ffe6-4c83-bad9-635aeba91586 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a72b372c8d0469e815fbe536b4cf9bb'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:08.362758 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-65b6510c-ffe6-4c83-bad9-635aeba91586 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a72b372c8d0469e815fbe536b4cf9bb'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:08.371140 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-65b6510c-ffe6-4c83-bad9-635aeba91586 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a72b372c8d0469e815fbe536b4cf9bb'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:08.450330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 878/1757] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:08 2022] DELETE /identity/v3/users/2a72b372c8d0469e815fbe536b4cf9bb => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:09.086616 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf062a-7d2a-424f-9f25-f03dfb3cd1ca service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:09.104019 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf062a-7d2a-424f-9f25-f03dfb3cd1ca service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:09.159763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e13ec-d3d8-4f80-845b-e1fa6b0de667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:09.160604 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e13ec-d3d8-4f80-845b-e1fa6b0de667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:09.160891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e13ec-d3d8-4f80-845b-e1fa6b0de667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:09.161168 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e13ec-d3d8-4f80-845b-e1fa6b0de667 admin admin] PATH_INFO: `/v3/projects/f102c3f973aa49729ab0898bb28cd3bd` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:09.162411 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e13ec-d3d8-4f80-845b-e1fa6b0de667 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f102c3f973aa49729ab0898bb28cd3bd)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:09.164318 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e13ec-d3d8-4f80-845b-e1fa6b0de667 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:09.261352 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-238e13ec-d3d8-4f80-845b-e1fa6b0de667 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f102c3f973aa49729ab0898bb28cd3bd', 'request_id': 'req-238e13ec-d3d8-4f80-845b-e1fa6b0de667'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:09.301507 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-238e13ec-d3d8-4f80-845b-e1fa6b0de667 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f102c3f973aa49729ab0898bb28cd3bd', 'request_id': 'req-238e13ec-d3d8-4f80-845b-e1fa6b0de667'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:09.306794 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-238e13ec-d3d8-4f80-845b-e1fa6b0de667 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f102c3f973aa49729ab0898bb28cd3bd', 'request_id': 'req-238e13ec-d3d8-4f80-845b-e1fa6b0de667'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:09.317857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-238e13ec-d3d8-4f80-845b-e1fa6b0de667 admin admin] The token cache is being invalidate because project f102c3f973aa49729ab0898bb28cd3bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:09.318172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-238e13ec-d3d8-4f80-845b-e1fa6b0de667 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:09.318791 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-238e13ec-d3d8-4f80-845b-e1fa6b0de667 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:09.320317 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 880/1758] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:09 2022] DELETE /identity/v3/projects/f102c3f973aa49729ab0898bb28cd3bd => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:09.325193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16968140-5e9e-498d-89f8-109c74ad3209 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:09.325656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16968140-5e9e-498d-89f8-109c74ad3209 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:09.325745 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16968140-5e9e-498d-89f8-109c74ad3209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:09.343607 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-16968140-5e9e-498d-89f8-109c74ad3209 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:09.344804 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-16968140-5e9e-498d-89f8-109c74ad3209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:09.403870 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 879/1759] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:09.411578 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c663198-b2d9-48cc-87cf-8c0bf09b8843 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:09.411871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c663198-b2d9-48cc-87cf-8c0bf09b8843 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:09.412154 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c663198-b2d9-48cc-87cf-8c0bf09b8843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:09.430471 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-8c663198-b2d9-48cc-87cf-8c0bf09b8843 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:09.432057 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-8c663198-b2d9-48cc-87cf-8c0bf09b8843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:09.439417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 881/1760] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:09.444234 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16968140-5e9e-498d-89f8-109c74ad3209 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:09.453915 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16968140-5e9e-498d-89f8-109c74ad3209 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:09.459222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587aac7-516f-4e2c-b314-3a6d3b18e69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:09.459870 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587aac7-516f-4e2c-b314-3a6d3b18e69f admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:09.460157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587aac7-516f-4e2c-b314-3a6d3b18e69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:09.460436 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587aac7-516f-4e2c-b314-3a6d3b18e69f admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:09.461200 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e587aac7-516f-4e2c-b314-3a6d3b18e69f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:17:09.461505 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e587aac7-516f-4e2c-b314-3a6d3b18e69f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:09.463133 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e587aac7-516f-4e2c-b314-3a6d3b18e69f admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:09.474172 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 880/1761] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 20:17:09.478222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c663198-b2d9-48cc-87cf-8c0bf09b8843 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:09.487189 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c663198-b2d9-48cc-87cf-8c0bf09b8843 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:09.491820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74077d-064d-4a3e-95c0-f8041ce95796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:09.492461 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74077d-064d-4a3e-95c0-f8041ce95796 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:09.493030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74077d-064d-4a3e-95c0-f8041ce95796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:09.493030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74077d-064d-4a3e-95c0-f8041ce95796 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:09.493697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74077d-064d-4a3e-95c0-f8041ce95796 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:09.495608 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74077d-064d-4a3e-95c0-f8041ce95796 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:09.599624 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 882/1762] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:09 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:09.606429 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587aac7-516f-4e2c-b314-3a6d3b18e69f admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:09.617648 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587aac7-516f-4e2c-b314-3a6d3b18e69f admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:09.687679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182de94-29f7-412d-b92d-0780a06915cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:09.688313 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182de94-29f7-412d-b92d-0780a06915cd admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:09.688602 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182de94-29f7-412d-b92d-0780a06915cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:09.688883 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182de94-29f7-412d-b92d-0780a06915cd admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:09.689595 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182de94-29f7-412d-b92d-0780a06915cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:09.691529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182de94-29f7-412d-b92d-0780a06915cd admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:09.759540 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 881/1763] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:09 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:09.769499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74077d-064d-4a3e-95c0-f8041ce95796 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:09.780539 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74077d-064d-4a3e-95c0-f8041ce95796 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:09.785210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb6b9a-f267-487d-abeb-5699049e16ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:09.786565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb6b9a-f267-487d-abeb-5699049e16ba admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:09.786848 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb6b9a-f267-487d-abeb-5699049e16ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:09.787118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb6b9a-f267-487d-abeb-5699049e16ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:09.787861 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb6b9a-f267-487d-abeb-5699049e16ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:09.789432 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb6b9a-f267-487d-abeb-5699049e16ba admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:09.800782 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 883/1764] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:09.804668 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182de94-29f7-412d-b92d-0780a06915cd admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:09.814739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182de94-29f7-412d-b92d-0780a06915cd admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:09.819647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef1bf2-f585-482f-9915-a791f6e2a897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:09.820375 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef1bf2-f585-482f-9915-a791f6e2a897 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:09.820666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef1bf2-f585-482f-9915-a791f6e2a897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:09.820947 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef1bf2-f585-482f-9915-a791f6e2a897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:09.821885 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef1bf2-f585-482f-9915-a791f6e2a897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:09.823474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef1bf2-f585-482f-9915-a791f6e2a897 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:09.837761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 882/1765] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:09.838962 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb6b9a-f267-487d-abeb-5699049e16ba admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:09.850661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb6b9a-f267-487d-abeb-5699049e16ba admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:09.855505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56aed5b-b692-4f94-a6c6-8bc563639069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:09.856444 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56aed5b-b692-4f94-a6c6-8bc563639069 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:09.856729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56aed5b-b692-4f94-a6c6-8bc563639069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:09.857015 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56aed5b-b692-4f94-a6c6-8bc563639069 admin admin] PATH_INFO: `/v3/projects/a405a056737f47d5947246746e031a72/users/2915e011cbff4b5fb345746f60e41243/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:09.873032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56aed5b-b692-4f94-a6c6-8bc563639069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a405a056737f47d5947246746e031a72, user_id=2915e011cbff4b5fb345746f60e41243, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:09.875775 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56aed5b-b692-4f94-a6c6-8bc563639069 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:09.976607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 884/1766] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:09 2022] PUT /identity/v3/projects/a405a056737f47d5947246746e031a72/users/2915e011cbff4b5fb345746f60e41243/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:09.989755 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601af27-4d87-4e24-bc03-9e9af18658ac None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:09.990050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601af27-4d87-4e24-bc03-9e9af18658ac None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:09.990392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601af27-4d87-4e24-bc03-9e9af18658ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:10.017034 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-d601af27-4d87-4e24-bc03-9e9af18658ac None None] MFA Rules not processed for user `2915e011cbff4b5fb345746f60e41243`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:10.018295 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-d601af27-4d87-4e24-bc03-9e9af18658ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:10.099568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 883/1767] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:10.104193 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56aed5b-b692-4f94-a6c6-8bc563639069 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:10.163386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56aed5b-b692-4f94-a6c6-8bc563639069 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:10.167945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197cca49-5105-43ce-8381-8e04cff1cfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:10.168589 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197cca49-5105-43ce-8381-8e04cff1cfa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:10.168871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197cca49-5105-43ce-8381-8e04cff1cfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:10.169143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197cca49-5105-43ce-8381-8e04cff1cfa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:10.169837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197cca49-5105-43ce-8381-8e04cff1cfa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:10.171820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197cca49-5105-43ce-8381-8e04cff1cfa9 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:10.251342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 885/1768] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:10 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:10.255824 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601af27-4d87-4e24-bc03-9e9af18658ac None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:10.266246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601af27-4d87-4e24-bc03-9e9af18658ac None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:10.321177 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e3eb9-2fa0-4b61-acc9-af7df19b6f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:10.321841 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e3eb9-2fa0-4b61-acc9-af7df19b6f5b admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:10.322152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e3eb9-2fa0-4b61-acc9-af7df19b6f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:10.322432 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e3eb9-2fa0-4b61-acc9-af7df19b6f5b admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:10.323130 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e3eb9-2fa0-4b61-acc9-af7df19b6f5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:10.325012 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e3eb9-2fa0-4b61-acc9-af7df19b6f5b admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:10.393275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 884/1769] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:10 2022] POST /identity/v3/users => generated 299 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:17:10.398007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197cca49-5105-43ce-8381-8e04cff1cfa9 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:10.412193 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197cca49-5105-43ce-8381-8e04cff1cfa9 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:10.417759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78114027-0cd4-4430-b5a7-dec4f688a31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:10.418453 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78114027-0cd4-4430-b5a7-dec4f688a31d admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:10.418736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78114027-0cd4-4430-b5a7-dec4f688a31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:10.419008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78114027-0cd4-4430-b5a7-dec4f688a31d admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:10.419747 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78114027-0cd4-4430-b5a7-dec4f688a31d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:10.421331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78114027-0cd4-4430-b5a7-dec4f688a31d admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:10.432404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 886/1770] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:10.436625 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e3eb9-2fa0-4b61-acc9-af7df19b6f5b admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:10.446537 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e3eb9-2fa0-4b61-acc9-af7df19b6f5b admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:10.451073 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e59b94-542b-4c86-b1d9-718f0dc2edbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:10.451943 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e59b94-542b-4c86-b1d9-718f0dc2edbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:10.452222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e59b94-542b-4c86-b1d9-718f0dc2edbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:10.452498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e59b94-542b-4c86-b1d9-718f0dc2edbb admin admin] PATH_INFO: `/v3/projects/afc4d736d59c4dadb0ef177942429bc0/users/f6b32b45c7ad4770b8c08bfd161fb3c3/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:10.467199 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e59b94-542b-4c86-b1d9-718f0dc2edbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc4d736d59c4dadb0ef177942429bc0, user_id=f6b32b45c7ad4770b8c08bfd161fb3c3, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:10.469830 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e59b94-542b-4c86-b1d9-718f0dc2edbb admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:10.547177 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 885/1771] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:10 2022] PUT /identity/v3/projects/afc4d736d59c4dadb0ef177942429bc0/users/f6b32b45c7ad4770b8c08bfd161fb3c3/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:10.551286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78114027-0cd4-4430-b5a7-dec4f688a31d admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:10.561256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78114027-0cd4-4430-b5a7-dec4f688a31d admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:10.615716 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a0b62-8399-421d-aecd-772333f3cd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:10.616375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a0b62-8399-421d-aecd-772333f3cd53 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:10.616662 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a0b62-8399-421d-aecd-772333f3cd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:10.616939 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a0b62-8399-421d-aecd-772333f3cd53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:10.617725 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a0b62-8399-421d-aecd-772333f3cd53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:10.619344 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a0b62-8399-421d-aecd-772333f3cd53 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:10.628812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 887/1772] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:10.633514 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e59b94-542b-4c86-b1d9-718f0dc2edbb admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:10.644307 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e59b94-542b-4c86-b1d9-718f0dc2edbb admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:10.649584 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76197fb-130d-4696-a74f-8dbd546cbc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:10.650680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76197fb-130d-4696-a74f-8dbd546cbc06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:10.651132 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76197fb-130d-4696-a74f-8dbd546cbc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:10.651539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76197fb-130d-4696-a74f-8dbd546cbc06 admin admin] PATH_INFO: `/v3/projects/afc4d736d59c4dadb0ef177942429bc0/users/f6b32b45c7ad4770b8c08bfd161fb3c3/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:10.653559 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76197fb-130d-4696-a74f-8dbd546cbc06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc4d736d59c4dadb0ef177942429bc0, user_id=f6b32b45c7ad4770b8c08bfd161fb3c3, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:10.656195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76197fb-130d-4696-a74f-8dbd546cbc06 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:10.667106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 886/1773] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:10 2022] PUT /identity/v3/projects/afc4d736d59c4dadb0ef177942429bc0/users/f6b32b45c7ad4770b8c08bfd161fb3c3/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:10.678158 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e1f7b-5ce6-4f7f-adca-32530bf52537 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:10.678454 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e1f7b-5ce6-4f7f-adca-32530bf52537 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:10.684058 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e1f7b-5ce6-4f7f-adca-32530bf52537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:10.704861 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-b10e1f7b-5ce6-4f7f-adca-32530bf52537 None None] MFA Rules not processed for user `f6b32b45c7ad4770b8c08bfd161fb3c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:10.706058 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b10e1f7b-5ce6-4f7f-adca-32530bf52537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:10.781817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 888/1774] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:10.793498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76197fb-130d-4696-a74f-8dbd546cbc06 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:10.794789 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f76197fb-130d-4696-a74f-8dbd546cbc06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:10.853969 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76197fb-130d-4696-a74f-8dbd546cbc06 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:10.857594 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03ed38-f27a-44e6-b3f7-ac045f172fe2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:10.858277 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03ed38-f27a-44e6-b3f7-ac045f172fe2 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:10.858566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03ed38-f27a-44e6-b3f7-ac045f172fe2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:10.858841 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03ed38-f27a-44e6-b3f7-ac045f172fe2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:10.867552 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03ed38-f27a-44e6-b3f7-ac045f172fe2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:10.868969 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03ed38-f27a-44e6-b3f7-ac045f172fe2 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:10.878162 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 887/1775] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:17:10.905832 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e1f7b-5ce6-4f7f-adca-32530bf52537 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:10.967681 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e1f7b-5ce6-4f7f-adca-32530bf52537 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:10.972635 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edfae1-2d49-405a-a5b6-94ae5d3d53b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:10.973538 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edfae1-2d49-405a-a5b6-94ae5d3d53b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:10.973829 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edfae1-2d49-405a-a5b6-94ae5d3d53b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:10.974528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edfae1-2d49-405a-a5b6-94ae5d3d53b5 admin admin] PATH_INFO: `/v3/users/2915e011cbff4b5fb345746f60e41243` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:10.975521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3edfae1-2d49-405a-a5b6-94ae5d3d53b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2915e011cbff4b5fb345746f60e41243)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:10.977718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3edfae1-2d49-405a-a5b6-94ae5d3d53b5 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:11.254196 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-f3edfae1-2d49-405a-a5b6-94ae5d3d53b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2915e011cbff4b5fb345746f60e41243'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:11.256359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-f3edfae1-2d49-405a-a5b6-94ae5d3d53b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2915e011cbff4b5fb345746f60e41243'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:11.257294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-f3edfae1-2d49-405a-a5b6-94ae5d3d53b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2915e011cbff4b5fb345746f60e41243'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:11.274394 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-f3edfae1-2d49-405a-a5b6-94ae5d3d53b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2915e011cbff4b5fb345746f60e41243'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:11.282749 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-f3edfae1-2d49-405a-a5b6-94ae5d3d53b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2915e011cbff4b5fb345746f60e41243'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:11.389711 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 889/1776] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:10 2022] DELETE /identity/v3/users/2915e011cbff4b5fb345746f60e41243 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:11.394545 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5016752-8a20-4ad7-a907-e3dcffc10b12 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:11.395108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5016752-8a20-4ad7-a907-e3dcffc10b12 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:11.395108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5016752-8a20-4ad7-a907-e3dcffc10b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:11.414439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-e5016752-8a20-4ad7-a907-e3dcffc10b12 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:11.423477 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-e5016752-8a20-4ad7-a907-e3dcffc10b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:11.501149 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 888/1777] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:11.511907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edfae1-2d49-405a-a5b6-94ae5d3d53b5 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:11.513588 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-f3edfae1-2d49-405a-a5b6-94ae5d3d53b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:11.593456 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edfae1-2d49-405a-a5b6-94ae5d3d53b5 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:11.597414 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e91dba-11b5-4de5-be9f-5656bf479168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:11.598180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e91dba-11b5-4de5-be9f-5656bf479168 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:11.598551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e91dba-11b5-4de5-be9f-5656bf479168 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:11.598898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e91dba-11b5-4de5-be9f-5656bf479168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:11.608460 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e91dba-11b5-4de5-be9f-5656bf479168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:11.610016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e91dba-11b5-4de5-be9f-5656bf479168 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:11.621314 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 890/1778] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:17:12.044024 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5016752-8a20-4ad7-a907-e3dcffc10b12 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:12.045924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-e5016752-8a20-4ad7-a907-e3dcffc10b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:12.061459 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5016752-8a20-4ad7-a907-e3dcffc10b12 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:12.069384 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f11c5-7b44-4403-92b9-3df6ec053ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:12.073142 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f11c5-7b44-4403-92b9-3df6ec053ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:12.075289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f11c5-7b44-4403-92b9-3df6ec053ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:12.075289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f11c5-7b44-4403-92b9-3df6ec053ab8 admin admin] PATH_INFO: `/v3/projects/a405a056737f47d5947246746e031a72` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:12.075439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f11c5-7b44-4403-92b9-3df6ec053ab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a405a056737f47d5947246746e031a72)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:12.078735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f11c5-7b44-4403-92b9-3df6ec053ab8 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:12.330838 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-739f11c5-7b44-4403-92b9-3df6ec053ab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a405a056737f47d5947246746e031a72', 'request_id': 'req-739f11c5-7b44-4403-92b9-3df6ec053ab8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:12.386606 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-739f11c5-7b44-4403-92b9-3df6ec053ab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a405a056737f47d5947246746e031a72', 'request_id': 'req-739f11c5-7b44-4403-92b9-3df6ec053ab8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:12.392711 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-739f11c5-7b44-4403-92b9-3df6ec053ab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a405a056737f47d5947246746e031a72', 'request_id': 'req-739f11c5-7b44-4403-92b9-3df6ec053ab8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:12.404097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-739f11c5-7b44-4403-92b9-3df6ec053ab8 admin admin] The token cache is being invalidate because project a405a056737f47d5947246746e031a72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:12.404680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-739f11c5-7b44-4403-92b9-3df6ec053ab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:12.405085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-739f11c5-7b44-4403-92b9-3df6ec053ab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:12.406864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 889/1779] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:12 2022] DELETE /identity/v3/projects/a405a056737f47d5947246746e031a72 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:12.410925 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e91dba-11b5-4de5-be9f-5656bf479168 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:12.421405 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e91dba-11b5-4de5-be9f-5656bf479168 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:12.479638 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124eadf-135b-4988-9dac-18a2adda4c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:12.480147 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124eadf-135b-4988-9dac-18a2adda4c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:12.480430 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124eadf-135b-4988-9dac-18a2adda4c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:12.480705 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124eadf-135b-4988-9dac-18a2adda4c02 admin admin] PATH_INFO: `/v3/users/f6b32b45c7ad4770b8c08bfd161fb3c3` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:12.481973 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124eadf-135b-4988-9dac-18a2adda4c02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6b32b45c7ad4770b8c08bfd161fb3c3)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:12.484106 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124eadf-135b-4988-9dac-18a2adda4c02 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:12.660561 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8124eadf-135b-4988-9dac-18a2adda4c02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6b32b45c7ad4770b8c08bfd161fb3c3'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:12.661298 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8124eadf-135b-4988-9dac-18a2adda4c02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6b32b45c7ad4770b8c08bfd161fb3c3'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:12.662816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8124eadf-135b-4988-9dac-18a2adda4c02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6b32b45c7ad4770b8c08bfd161fb3c3'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:12.676296 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8124eadf-135b-4988-9dac-18a2adda4c02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6b32b45c7ad4770b8c08bfd161fb3c3'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:12.693735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8124eadf-135b-4988-9dac-18a2adda4c02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6b32b45c7ad4770b8c08bfd161fb3c3'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:12.760319 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 891/1780] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:12 2022] DELETE /identity/v3/users/f6b32b45c7ad4770b8c08bfd161fb3c3 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:13.179712 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f11c5-7b44-4403-92b9-3df6ec053ab8 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:13.181257 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-739f11c5-7b44-4403-92b9-3df6ec053ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:13.203703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f11c5-7b44-4403-92b9-3df6ec053ab8 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:13.268296 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd3275-7371-408b-a88a-8b5137e6b2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:13.269164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd3275-7371-408b-a88a-8b5137e6b2ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:13.269447 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd3275-7371-408b-a88a-8b5137e6b2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:13.269721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd3275-7371-408b-a88a-8b5137e6b2ca admin admin] PATH_INFO: `/v3/projects/afc4d736d59c4dadb0ef177942429bc0` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:13.271042 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd3275-7371-408b-a88a-8b5137e6b2ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=afc4d736d59c4dadb0ef177942429bc0)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:13.276546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd3275-7371-408b-a88a-8b5137e6b2ca admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:13.376331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f7bd3275-7371-408b-a88a-8b5137e6b2ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afc4d736d59c4dadb0ef177942429bc0', 'request_id': 'req-f7bd3275-7371-408b-a88a-8b5137e6b2ca'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:13.439710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f7bd3275-7371-408b-a88a-8b5137e6b2ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afc4d736d59c4dadb0ef177942429bc0', 'request_id': 'req-f7bd3275-7371-408b-a88a-8b5137e6b2ca'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:13.447315 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f7bd3275-7371-408b-a88a-8b5137e6b2ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afc4d736d59c4dadb0ef177942429bc0', 'request_id': 'req-f7bd3275-7371-408b-a88a-8b5137e6b2ca'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:13.459448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f7bd3275-7371-408b-a88a-8b5137e6b2ca admin admin] The token cache is being invalidate because project afc4d736d59c4dadb0ef177942429bc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:13.459734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f7bd3275-7371-408b-a88a-8b5137e6b2ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:13.460397 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f7bd3275-7371-408b-a88a-8b5137e6b2ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:13.461941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 890/1781] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:13 2022] DELETE /identity/v3/projects/afc4d736d59c4dadb0ef177942429bc0 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:13.466760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d7dbd-d8bb-471e-bfb9-52a4c8c228f7 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:13.467218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d7dbd-d8bb-471e-bfb9-52a4c8c228f7 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:13.467308 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d7dbd-d8bb-471e-bfb9-52a4c8c228f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:13.484547 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-f23d7dbd-d8bb-471e-bfb9-52a4c8c228f7 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:13.485735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-f23d7dbd-d8bb-471e-bfb9-52a4c8c228f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:13.541639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 892/1782] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:13.549141 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2264b4-97ec-4822-82ae-f765ca7e20e2 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:13.549433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2264b4-97ec-4822-82ae-f765ca7e20e2 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:13.549706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2264b4-97ec-4822-82ae-f765ca7e20e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:13.567702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-0a2264b4-97ec-4822-82ae-f765ca7e20e2 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:13.568876 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0a2264b4-97ec-4822-82ae-f765ca7e20e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:13.577452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 891/1783] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:13.581585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d7dbd-d8bb-471e-bfb9-52a4c8c228f7 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:13.591027 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d7dbd-d8bb-471e-bfb9-52a4c8c228f7 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:13.595656 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef973037-b58f-4cc4-a79b-c013e26011ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:13.603312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef973037-b58f-4cc4-a79b-c013e26011ea admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:13.603312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef973037-b58f-4cc4-a79b-c013e26011ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:13.603312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef973037-b58f-4cc4-a79b-c013e26011ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:13.603312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef973037-b58f-4cc4-a79b-c013e26011ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:17:13.603312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef973037-b58f-4cc4-a79b-c013e26011ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:13.603312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef973037-b58f-4cc4-a79b-c013e26011ea admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:13.611275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 893/1784] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:17:13.615597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2264b4-97ec-4822-82ae-f765ca7e20e2 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:13.629662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2264b4-97ec-4822-82ae-f765ca7e20e2 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:13.635531 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66e334-bee3-4bb4-b388-94ec16030b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:13.635795 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66e334-bee3-4bb4-b388-94ec16030b30 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:13.635937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66e334-bee3-4bb4-b388-94ec16030b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:13.636206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66e334-bee3-4bb4-b388-94ec16030b30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:13.636909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66e334-bee3-4bb4-b388-94ec16030b30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:13.638817 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66e334-bee3-4bb4-b388-94ec16030b30 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:13.713011 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 892/1785] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:13 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:17:13.714158 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef973037-b58f-4cc4-a79b-c013e26011ea admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:13.724469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef973037-b58f-4cc4-a79b-c013e26011ea admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:13.797101 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351e0c7-adbd-42a6-96a4-6fb0bd617f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:13.797807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351e0c7-adbd-42a6-96a4-6fb0bd617f6b admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:13.798103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351e0c7-adbd-42a6-96a4-6fb0bd617f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:13.798406 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351e0c7-adbd-42a6-96a4-6fb0bd617f6b admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:13.799252 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351e0c7-adbd-42a6-96a4-6fb0bd617f6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:13.801440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351e0c7-adbd-42a6-96a4-6fb0bd617f6b admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:13.891604 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 894/1786] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:13 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:13.895805 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66e334-bee3-4bb4-b388-94ec16030b30 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:13.907060 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66e334-bee3-4bb4-b388-94ec16030b30 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:13.916007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08dc9a-11d1-435f-8920-66770c47effc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:13.916007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08dc9a-11d1-435f-8920-66770c47effc admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:13.916007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08dc9a-11d1-435f-8920-66770c47effc admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:13.916007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08dc9a-11d1-435f-8920-66770c47effc admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:13.916313 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08dc9a-11d1-435f-8920-66770c47effc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:13.916313 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08dc9a-11d1-435f-8920-66770c47effc admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:13.936286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 893/1787] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:13.939952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351e0c7-adbd-42a6-96a4-6fb0bd617f6b admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:13.949694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351e0c7-adbd-42a6-96a4-6fb0bd617f6b admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:13.954410 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347e4cd-3947-433a-ab9f-246411c5007d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:13.955068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347e4cd-3947-433a-ab9f-246411c5007d admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:13.955357 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347e4cd-3947-433a-ab9f-246411c5007d admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:13.955638 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347e4cd-3947-433a-ab9f-246411c5007d admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:13.956379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347e4cd-3947-433a-ab9f-246411c5007d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:13.957960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347e4cd-3947-433a-ab9f-246411c5007d admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:13.968253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 895/1788] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:13.972554 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08dc9a-11d1-435f-8920-66770c47effc admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:13.983883 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08dc9a-11d1-435f-8920-66770c47effc admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:13.988877 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc5c4c-308c-4c7d-a4a1-6d63b8ad3bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:13.989679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc5c4c-308c-4c7d-a4a1-6d63b8ad3bac admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:13.989962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc5c4c-308c-4c7d-a4a1-6d63b8ad3bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:13.990325 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc5c4c-308c-4c7d-a4a1-6d63b8ad3bac admin admin] PATH_INFO: `/v3/projects/85cf60ccc83946d9b53672f9667dfe8c/users/972a219d96de4a5f886b6ef66171d87d/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:14.006744 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc5c4c-308c-4c7d-a4a1-6d63b8ad3bac admin admin] RBAC: Authorizing `identity:create_grant(project_id=85cf60ccc83946d9b53672f9667dfe8c, user_id=972a219d96de4a5f886b6ef66171d87d, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:14.009712 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc5c4c-308c-4c7d-a4a1-6d63b8ad3bac admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:14.137959 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 894/1789] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:13 2022] PUT /identity/v3/projects/85cf60ccc83946d9b53672f9667dfe8c/users/972a219d96de4a5f886b6ef66171d87d/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:14.149464 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4355c-2950-442c-a797-f8a9c39c812b None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:14.149853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4355c-2950-442c-a797-f8a9c39c812b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:14.150238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4355c-2950-442c-a797-f8a9c39c812b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:14.180606 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-b6a4355c-2950-442c-a797-f8a9c39c812b None None] MFA Rules not processed for user `972a219d96de4a5f886b6ef66171d87d`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:14.181805 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b6a4355c-2950-442c-a797-f8a9c39c812b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:14.300643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 896/1790] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:14.305911 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc5c4c-308c-4c7d-a4a1-6d63b8ad3bac admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:14.408377 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc5c4c-308c-4c7d-a4a1-6d63b8ad3bac admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:14.448365 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f41a7-efe0-43a7-bf46-4beecfae4ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:14.450393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f41a7-efe0-43a7-bf46-4beecfae4ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:14.450684 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f41a7-efe0-43a7-bf46-4beecfae4ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:14.450967 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f41a7-efe0-43a7-bf46-4beecfae4ca6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:14.453341 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f41a7-efe0-43a7-bf46-4beecfae4ca6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:14.455291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f41a7-efe0-43a7-bf46-4beecfae4ca6 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:14.534303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 895/1791] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:14 2022] POST /identity/v3/projects => generated 379 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:14.538777 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4355c-2950-442c-a797-f8a9c39c812b None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:14.555682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4355c-2950-442c-a797-f8a9c39c812b None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:14.691279 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6627b-fd08-4522-bbcc-9a7da60c0118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:14.692200 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6627b-fd08-4522-bbcc-9a7da60c0118 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:14.692499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6627b-fd08-4522-bbcc-9a7da60c0118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:14.692754 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6627b-fd08-4522-bbcc-9a7da60c0118 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:14.693718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6627b-fd08-4522-bbcc-9a7da60c0118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:14.696911 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6627b-fd08-4522-bbcc-9a7da60c0118 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:14.836952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 897/1792] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:14 2022] POST /identity/v3/users => generated 296 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:17:14.841025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f41a7-efe0-43a7-bf46-4beecfae4ca6 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:14.854269 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f41a7-efe0-43a7-bf46-4beecfae4ca6 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:14.875258 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62868d47-5213-4f11-a0af-c719083e4565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:14.875941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62868d47-5213-4f11-a0af-c719083e4565 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:14.876230 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62868d47-5213-4f11-a0af-c719083e4565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:14.876511 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62868d47-5213-4f11-a0af-c719083e4565 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:14.877247 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62868d47-5213-4f11-a0af-c719083e4565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:14.878881 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62868d47-5213-4f11-a0af-c719083e4565 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:14.894575 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 896/1793] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:14.895951 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6627b-fd08-4522-bbcc-9a7da60c0118 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:14.910239 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6627b-fd08-4522-bbcc-9a7da60c0118 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:14.916916 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cd772-b33c-412e-a434-0251a5a021bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:14.918300 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cd772-b33c-412e-a434-0251a5a021bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:14.918663 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cd772-b33c-412e-a434-0251a5a021bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:14.919016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cd772-b33c-412e-a434-0251a5a021bc admin admin] PATH_INFO: `/v3/projects/ca49683347ca43abb4db836e47e8fe85/users/30c259d145e143cfad915d6e3b6f17dc/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:14.972913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6cd772-b33c-412e-a434-0251a5a021bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca49683347ca43abb4db836e47e8fe85, user_id=30c259d145e143cfad915d6e3b6f17dc, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:14.981064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6cd772-b33c-412e-a434-0251a5a021bc admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:15.109082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 898/1794] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:14 2022] PUT /identity/v3/projects/ca49683347ca43abb4db836e47e8fe85/users/30c259d145e143cfad915d6e3b6f17dc/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:15.110500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62868d47-5213-4f11-a0af-c719083e4565 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:15.127699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62868d47-5213-4f11-a0af-c719083e4565 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:15.227653 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48b7e0-7686-474b-8478-3ca3bfebba50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:15.228387 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48b7e0-7686-474b-8478-3ca3bfebba50 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:15.235761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48b7e0-7686-474b-8478-3ca3bfebba50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:15.236332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48b7e0-7686-474b-8478-3ca3bfebba50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:15.237758 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48b7e0-7686-474b-8478-3ca3bfebba50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:15.239864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48b7e0-7686-474b-8478-3ca3bfebba50 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:15.256138 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 897/1795] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:15.260498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cd772-b33c-412e-a434-0251a5a021bc admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:15.271991 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cd772-b33c-412e-a434-0251a5a021bc admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:15.278105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd30c6d-8c05-496c-a9dc-06e32f9d0216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:15.278578 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd30c6d-8c05-496c-a9dc-06e32f9d0216 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:15.278941 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd30c6d-8c05-496c-a9dc-06e32f9d0216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:15.279184 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd30c6d-8c05-496c-a9dc-06e32f9d0216 admin admin] PATH_INFO: `/v3/projects/ca49683347ca43abb4db836e47e8fe85/users/30c259d145e143cfad915d6e3b6f17dc/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:15.281726 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd30c6d-8c05-496c-a9dc-06e32f9d0216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca49683347ca43abb4db836e47e8fe85, user_id=30c259d145e143cfad915d6e3b6f17dc, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:15.284688 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd30c6d-8c05-496c-a9dc-06e32f9d0216 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:15.298337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 899/1796] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:15 2022] PUT /identity/v3/projects/ca49683347ca43abb4db836e47e8fe85/users/30c259d145e143cfad915d6e3b6f17dc/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:15.311254 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8c524-87cf-47d9-986a-7e2bf7caad1b None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:15.311697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8c524-87cf-47d9-986a-7e2bf7caad1b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:15.312063 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8c524-87cf-47d9-986a-7e2bf7caad1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:15.352618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-f5e8c524-87cf-47d9-986a-7e2bf7caad1b None None] MFA Rules not processed for user `30c259d145e143cfad915d6e3b6f17dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:15.355532 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f5e8c524-87cf-47d9-986a-7e2bf7caad1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:15.519766 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 898/1797] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:15.543451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd30c6d-8c05-496c-a9dc-06e32f9d0216 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:15.545214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-ffd30c6d-8c05-496c-a9dc-06e32f9d0216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:15.634409 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd30c6d-8c05-496c-a9dc-06e32f9d0216 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:15.644295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3ae40-8718-4dfb-97d6-2b5e0285ddfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:15.645148 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3ae40-8718-4dfb-97d6-2b5e0285ddfa service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:15.646587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3ae40-8718-4dfb-97d6-2b5e0285ddfa service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:15.646876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3ae40-8718-4dfb-97d6-2b5e0285ddfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:15.678282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3ae40-8718-4dfb-97d6-2b5e0285ddfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:15.679739 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3ae40-8718-4dfb-97d6-2b5e0285ddfa service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:15.716203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 900/1798] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:17:16.362837 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d7458-61f5-4cda-8bc2-eb5c7ce6358b None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:16.363204 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d7458-61f5-4cda-8bc2-eb5c7ce6358b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:16.363568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d7458-61f5-4cda-8bc2-eb5c7ce6358b None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:16.365700 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 899/1799] 10.222.0.65 () {54 vars in 921 bytes} [Tue May 10 20:17:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:17:16.377886 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395aeea7-7f22-4d2b-826d-74b32ad4cc01 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:16.378214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395aeea7-7f22-4d2b-826d-74b32ad4cc01 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:16.378977 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395aeea7-7f22-4d2b-826d-74b32ad4cc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:16.404699 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-395aeea7-7f22-4d2b-826d-74b32ad4cc01 None None] MFA Rules not processed for user `ae922b2da4814d10bed1fa345700ac04`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:16.406188 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-395aeea7-7f22-4d2b-826d-74b32ad4cc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:16.488333 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 901/1800] 10.222.0.65 () {58 vars in 1018 bytes} [Tue May 10 20:17:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:16.527928 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d7458-61f5-4cda-8bc2-eb5c7ce6358b None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:16.530684 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-529d7458-61f5-4cda-8bc2-eb5c7ce6358b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:16.583463 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d7458-61f5-4cda-8bc2-eb5c7ce6358b None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:16.594128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ab170-72f0-4a92-8df8-32bb1392b63e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:16.595850 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ab170-72f0-4a92-8df8-32bb1392b63e service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:16.599655 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ab170-72f0-4a92-8df8-32bb1392b63e service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:16.599655 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ab170-72f0-4a92-8df8-32bb1392b63e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:16.615690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ab170-72f0-4a92-8df8-32bb1392b63e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:16.621335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ab170-72f0-4a92-8df8-32bb1392b63e service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:16.665097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 900/1801] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:17:16.802787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395aeea7-7f22-4d2b-826d-74b32ad4cc01 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:16.820437 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395aeea7-7f22-4d2b-826d-74b32ad4cc01 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:16.822002 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ab170-72f0-4a92-8df8-32bb1392b63e service nova] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:16.823646 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-423ab170-72f0-4a92-8df8-32bb1392b63e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:16.827241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36283421-e385-48d6-9298-ab97b36201de tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30c259d145e143cfad915d6e3b6f17dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca49683347ca43abb4db836e47e8fe85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:16.828204 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36283421-e385-48d6-9298-ab97b36201de tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:16.828761 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36283421-e385-48d6-9298-ab97b36201de tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:16.829474 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36283421-e385-48d6-9298-ab97b36201de tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:16.830554 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36283421-e385-48d6-9298-ab97b36201de tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:17:16.831069 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36283421-e385-48d6-9298-ab97b36201de tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:16.833112 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36283421-e385-48d6-9298-ab97b36201de tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:16.847759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 902/1802] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:17:16.857181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36283421-e385-48d6-9298-ab97b36201de tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:16.892269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36283421-e385-48d6-9298-ab97b36201de tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:16.910829 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafc209-dd29-4ffe-9abf-a8e03c581817 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30c259d145e143cfad915d6e3b6f17dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca49683347ca43abb4db836e47e8fe85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:16.913800 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafc209-dd29-4ffe-9abf-a8e03c581817 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:16.918589 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafc209-dd29-4ffe-9abf-a8e03c581817 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:16.918589 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafc209-dd29-4ffe-9abf-a8e03c581817 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:16.918820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafc209-dd29-4ffe-9abf-a8e03c581817 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:16.925851 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafc209-dd29-4ffe-9abf-a8e03c581817 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:17.000967 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ab170-72f0-4a92-8df8-32bb1392b63e service nova] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:17.014234 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57614d11-13ff-44a4-9de2-5c03cd9e224c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:17.015139 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57614d11-13ff-44a4-9de2-5c03cd9e224c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:17.015429 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57614d11-13ff-44a4-9de2-5c03cd9e224c admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:17.015707 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57614d11-13ff-44a4-9de2-5c03cd9e224c admin admin] PATH_INFO: `/v3/users/9996d2c2b37e4a19a80211b79ee88a2d` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:17.020946 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57614d11-13ff-44a4-9de2-5c03cd9e224c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9996d2c2b37e4a19a80211b79ee88a2d)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:17.023510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57614d11-13ff-44a4-9de2-5c03cd9e224c admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:17.072067 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 903/1803] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:16 2022] POST /identity/v3/projects => generated 354 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:17.077704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafc209-dd29-4ffe-9abf-a8e03c581817 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:17.212877 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafc209-dd29-4ffe-9abf-a8e03c581817 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:17.228238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639aa513-7140-4add-913d-06f088b68441 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30c259d145e143cfad915d6e3b6f17dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca49683347ca43abb4db836e47e8fe85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:17.232031 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639aa513-7140-4add-913d-06f088b68441 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:17.234530 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639aa513-7140-4add-913d-06f088b68441 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:17.235209 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639aa513-7140-4add-913d-06f088b68441 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:17.236186 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639aa513-7140-4add-913d-06f088b68441 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:17:17.237568 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639aa513-7140-4add-913d-06f088b68441 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:17.241779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639aa513-7140-4add-913d-06f088b68441 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:17.260247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 904/1804] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:17:17.326190 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-57614d11-13ff-44a4-9de2-5c03cd9e224c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9996d2c2b37e4a19a80211b79ee88a2d'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:17.326932 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-57614d11-13ff-44a4-9de2-5c03cd9e224c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9996d2c2b37e4a19a80211b79ee88a2d'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:17.348176 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-57614d11-13ff-44a4-9de2-5c03cd9e224c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9996d2c2b37e4a19a80211b79ee88a2d'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:17.349485 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-57614d11-13ff-44a4-9de2-5c03cd9e224c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9996d2c2b37e4a19a80211b79ee88a2d'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:17.358488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-57614d11-13ff-44a4-9de2-5c03cd9e224c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9996d2c2b37e4a19a80211b79ee88a2d'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:17.413661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639aa513-7140-4add-913d-06f088b68441 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:17.417006 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-639aa513-7140-4add-913d-06f088b68441 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:17.439196 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639aa513-7140-4add-913d-06f088b68441 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:17.448325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028c510-296a-4b7f-96b4-3f6d4987a753 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30c259d145e143cfad915d6e3b6f17dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca49683347ca43abb4db836e47e8fe85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:17.449377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028c510-296a-4b7f-96b4-3f6d4987a753 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:17.450148 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028c510-296a-4b7f-96b4-3f6d4987a753 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:17.450578 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028c510-296a-4b7f-96b4-3f6d4987a753 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] PATH_INFO: `/v3/projects/1b4058ea964a497eaa719519a6bf32c0` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:17.452117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028c510-296a-4b7f-96b4-3f6d4987a753 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1b4058ea964a497eaa719519a6bf32c0)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:17.454440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028c510-296a-4b7f-96b4-3f6d4987a753 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:17.498794 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 901/1805] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:16 2022] DELETE /identity/v3/users/9996d2c2b37e4a19a80211b79ee88a2d => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:17.503331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6757557-aec6-4b76-9f20-02bd751f204a None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:17.503626 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6757557-aec6-4b76-9f20-02bd751f204a None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:17.503907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6757557-aec6-4b76-9f20-02bd751f204a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:17.528446 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-b6757557-aec6-4b76-9f20-02bd751f204a None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:17.533915 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b6757557-aec6-4b76-9f20-02bd751f204a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:17.630387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5028c510-296a-4b7f-96b4-3f6d4987a753 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b4058ea964a497eaa719519a6bf32c0', 'request_id': 'req-5028c510-296a-4b7f-96b4-3f6d4987a753'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:17.641805 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 902/1806] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:17.653456 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6757557-aec6-4b76-9f20-02bd751f204a None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:17.655386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b6757557-aec6-4b76-9f20-02bd751f204a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:17.668533 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6757557-aec6-4b76-9f20-02bd751f204a None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:17.709643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5028c510-296a-4b7f-96b4-3f6d4987a753 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b4058ea964a497eaa719519a6bf32c0', 'request_id': 'req-5028c510-296a-4b7f-96b4-3f6d4987a753'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:17.717018 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5028c510-296a-4b7f-96b4-3f6d4987a753 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b4058ea964a497eaa719519a6bf32c0', 'request_id': 'req-5028c510-296a-4b7f-96b4-3f6d4987a753'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:17.731164 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5028c510-296a-4b7f-96b4-3f6d4987a753 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] The token cache is being invalidate because project 1b4058ea964a497eaa719519a6bf32c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:17.732153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5028c510-296a-4b7f-96b4-3f6d4987a753 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:17.732539 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5028c510-296a-4b7f-96b4-3f6d4987a753 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:17.734564 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 905/1807] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:17 2022] DELETE /identity/v3/projects/1b4058ea964a497eaa719519a6bf32c0 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:17.735493 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028c510-296a-4b7f-96b4-3f6d4987a753 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:17.751345 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-5028c510-296a-4b7f-96b4-3f6d4987a753 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:17.770926 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c5921-b150-40f2-845a-1b56fcc1eb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:17.772866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c5921-b150-40f2-845a-1b56fcc1eb8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:17.773637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c5921-b150-40f2-845a-1b56fcc1eb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:17.774180 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c5921-b150-40f2-845a-1b56fcc1eb8e admin admin] PATH_INFO: `/v3/users/fa7fbd21a8a447adb2775a24e9894541` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:17.776991 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c5921-b150-40f2-845a-1b56fcc1eb8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa7fbd21a8a447adb2775a24e9894541)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:17.780358 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c5921-b150-40f2-845a-1b56fcc1eb8e admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:17.881745 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028c510-296a-4b7f-96b4-3f6d4987a753 tempest-VolumeQuotaClassesTest-714907104 tempest-VolumeQuotaClassesTest-714907104-project-admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:17.894627 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b499c45-51bb-4bf7-a885-0228cf6653fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:17.894627 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b499c45-51bb-4bf7-a885-0228cf6653fb service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:17.894627 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b499c45-51bb-4bf7-a885-0228cf6653fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:17.894627 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b499c45-51bb-4bf7-a885-0228cf6653fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:17.895172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-8b499c45-51bb-4bf7-a885-0228cf6653fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:17.911045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b499c45-51bb-4bf7-a885-0228cf6653fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:17.912505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b499c45-51bb-4bf7-a885-0228cf6653fb service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:17.938996 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 906/1808] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:17:18.024289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-bc2c5921-b150-40f2-845a-1b56fcc1eb8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa7fbd21a8a447adb2775a24e9894541'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:18.025148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-bc2c5921-b150-40f2-845a-1b56fcc1eb8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa7fbd21a8a447adb2775a24e9894541'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:18.062075 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-bc2c5921-b150-40f2-845a-1b56fcc1eb8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa7fbd21a8a447adb2775a24e9894541'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:18.067262 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-bc2c5921-b150-40f2-845a-1b56fcc1eb8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa7fbd21a8a447adb2775a24e9894541'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:18.082691 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-bc2c5921-b150-40f2-845a-1b56fcc1eb8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa7fbd21a8a447adb2775a24e9894541'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:18.186816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 903/1809] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:17 2022] DELETE /identity/v3/users/fa7fbd21a8a447adb2775a24e9894541 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:18.192141 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdecde8-d254-434d-b9b4-ed11e45532f0 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:18.192575 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdecde8-d254-434d-b9b4-ed11e45532f0 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:18.192952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdecde8-d254-434d-b9b4-ed11e45532f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:18.226682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-0cdecde8-d254-434d-b9b4-ed11e45532f0 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:18.230948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-0cdecde8-d254-434d-b9b4-ed11e45532f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:18.345764 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 907/1810] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:18.367637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c5921-b150-40f2-845a-1b56fcc1eb8e admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:18.371054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-bc2c5921-b150-40f2-845a-1b56fcc1eb8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:18.470382 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c5921-b150-40f2-845a-1b56fcc1eb8e admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:18.476124 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a8c5a-c370-46b8-8f87-700a5e6d01ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:18.476908 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a8c5a-c370-46b8-8f87-700a5e6d01ea service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:18.477192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a8c5a-c370-46b8-8f87-700a5e6d01ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:18.478512 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a8c5a-c370-46b8-8f87-700a5e6d01ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:18.490372 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a8c5a-c370-46b8-8f87-700a5e6d01ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:18.492701 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a8c5a-c370-46b8-8f87-700a5e6d01ea service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:18.505494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 904/1811] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:17:18.569282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdecde8-d254-434d-b9b4-ed11e45532f0 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:18.570893 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-0cdecde8-d254-434d-b9b4-ed11e45532f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:18.592495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdecde8-d254-434d-b9b4-ed11e45532f0 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:18.606768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3866760-03d6-44e1-8b95-ebb00bb5e453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:18.608024 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3866760-03d6-44e1-8b95-ebb00bb5e453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:18.608557 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3866760-03d6-44e1-8b95-ebb00bb5e453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:18.608842 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3866760-03d6-44e1-8b95-ebb00bb5e453 admin admin] PATH_INFO: `/v3/users/972a219d96de4a5f886b6ef66171d87d` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:18.614855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3866760-03d6-44e1-8b95-ebb00bb5e453 admin admin] RBAC: Authorizing `identity:delete_user(user_id=972a219d96de4a5f886b6ef66171d87d)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:18.617083 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3866760-03d6-44e1-8b95-ebb00bb5e453 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:18.899337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a8c5a-c370-46b8-8f87-700a5e6d01ea service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:18.902194 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-179a8c5a-c370-46b8-8f87-700a5e6d01ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:18.916946 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a8c5a-c370-46b8-8f87-700a5e6d01ea service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:18.923280 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b52d9-1518-44fa-bf6e-5757796b194b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:18.928771 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b52d9-1518-44fa-bf6e-5757796b194b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:18.929398 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b52d9-1518-44fa-bf6e-5757796b194b admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:18.929923 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b52d9-1518-44fa-bf6e-5757796b194b admin admin] PATH_INFO: `/v3/projects/a520b50d4b47438b827bd2cfc5432270` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:18.931865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b52d9-1518-44fa-bf6e-5757796b194b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a520b50d4b47438b827bd2cfc5432270)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:18.934728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b52d9-1518-44fa-bf6e-5757796b194b admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:18.989221 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c3866760-03d6-44e1-8b95-ebb00bb5e453 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '972a219d96de4a5f886b6ef66171d87d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:18.989925 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c3866760-03d6-44e1-8b95-ebb00bb5e453 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '972a219d96de4a5f886b6ef66171d87d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:18.990650 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c3866760-03d6-44e1-8b95-ebb00bb5e453 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '972a219d96de4a5f886b6ef66171d87d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:19.013717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c3866760-03d6-44e1-8b95-ebb00bb5e453 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '972a219d96de4a5f886b6ef66171d87d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:19.032580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c3866760-03d6-44e1-8b95-ebb00bb5e453 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '972a219d96de4a5f886b6ef66171d87d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:19.122781 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 908/1812] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:18 2022] DELETE /identity/v3/users/972a219d96de4a5f886b6ef66171d87d => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:19.127896 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221aeb5f-5b53-4f66-9b60-3585b47a3eac None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:19.128439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221aeb5f-5b53-4f66-9b60-3585b47a3eac None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:19.129921 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221aeb5f-5b53-4f66-9b60-3585b47a3eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:19.151610 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-902b52d9-1518-44fa-bf6e-5757796b194b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a520b50d4b47438b827bd2cfc5432270', 'request_id': 'req-902b52d9-1518-44fa-bf6e-5757796b194b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:19.182653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-221aeb5f-5b53-4f66-9b60-3585b47a3eac None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:19.183655 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-221aeb5f-5b53-4f66-9b60-3585b47a3eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:19.217923 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-902b52d9-1518-44fa-bf6e-5757796b194b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a520b50d4b47438b827bd2cfc5432270', 'request_id': 'req-902b52d9-1518-44fa-bf6e-5757796b194b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:19.225116 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-902b52d9-1518-44fa-bf6e-5757796b194b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a520b50d4b47438b827bd2cfc5432270', 'request_id': 'req-902b52d9-1518-44fa-bf6e-5757796b194b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:19.253340 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-902b52d9-1518-44fa-bf6e-5757796b194b admin admin] The token cache is being invalidate because project a520b50d4b47438b827bd2cfc5432270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:19.253637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-902b52d9-1518-44fa-bf6e-5757796b194b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:19.254828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-902b52d9-1518-44fa-bf6e-5757796b194b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:19.257296 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 905/1813] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:18 2022] DELETE /identity/v3/projects/a520b50d4b47438b827bd2cfc5432270 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:19.263316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b52d9-1518-44fa-bf6e-5757796b194b admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:19.266877 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-902b52d9-1518-44fa-bf6e-5757796b194b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:19.305279 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 909/1814] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:19.340565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221aeb5f-5b53-4f66-9b60-3585b47a3eac None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:19.342982 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-221aeb5f-5b53-4f66-9b60-3585b47a3eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:19.392145 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b52d9-1518-44fa-bf6e-5757796b194b admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:19.401704 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5087d-8d49-4a82-95d7-8c673325ad25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:19.402455 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5087d-8d49-4a82-95d7-8c673325ad25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:19.402780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5087d-8d49-4a82-95d7-8c673325ad25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:19.403061 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5087d-8d49-4a82-95d7-8c673325ad25 admin admin] PATH_INFO: `/v3/users/0bba3a68755b491c9589786d2ffaac3f` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:19.405578 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5087d-8d49-4a82-95d7-8c673325ad25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bba3a68755b491c9589786d2ffaac3f)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:19.408678 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5087d-8d49-4a82-95d7-8c673325ad25 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:19.461804 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221aeb5f-5b53-4f66-9b60-3585b47a3eac None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:19.465767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de92ac-050a-44c6-a2f7-1f83c20cdf84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:19.470810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de92ac-050a-44c6-a2f7-1f83c20cdf84 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:19.470810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de92ac-050a-44c6-a2f7-1f83c20cdf84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:19.470810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de92ac-050a-44c6-a2f7-1f83c20cdf84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:19.477901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de92ac-050a-44c6-a2f7-1f83c20cdf84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:19.479453 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de92ac-050a-44c6-a2f7-1f83c20cdf84 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:19.495253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 910/1815] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:17:19.571988 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de92ac-050a-44c6-a2f7-1f83c20cdf84 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:19.573609 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a5de92ac-050a-44c6-a2f7-1f83c20cdf84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:19.587399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de92ac-050a-44c6-a2f7-1f83c20cdf84 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:19.592701 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533302f-cc52-4cac-98f7-bdc4dd16c6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:19.593693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533302f-cc52-4cac-98f7-bdc4dd16c6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:19.597803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533302f-cc52-4cac-98f7-bdc4dd16c6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:19.597803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533302f-cc52-4cac-98f7-bdc4dd16c6fd admin admin] PATH_INFO: `/v3/projects/1843d1fa62bd4ee2b05c46b7000f36b2` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:19.597803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533302f-cc52-4cac-98f7-bdc4dd16c6fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1843d1fa62bd4ee2b05c46b7000f36b2)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:19.598018 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533302f-cc52-4cac-98f7-bdc4dd16c6fd admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:19.670664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-eee5087d-8d49-4a82-95d7-8c673325ad25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bba3a68755b491c9589786d2ffaac3f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:19.671332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-eee5087d-8d49-4a82-95d7-8c673325ad25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bba3a68755b491c9589786d2ffaac3f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:19.708994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-eee5087d-8d49-4a82-95d7-8c673325ad25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bba3a68755b491c9589786d2ffaac3f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:19.710597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-eee5087d-8d49-4a82-95d7-8c673325ad25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bba3a68755b491c9589786d2ffaac3f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:19.732985 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-eee5087d-8d49-4a82-95d7-8c673325ad25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bba3a68755b491c9589786d2ffaac3f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:19.939405 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 906/1816] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:19 2022] DELETE /identity/v3/users/0bba3a68755b491c9589786d2ffaac3f => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:19.955656 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e533302f-cc52-4cac-98f7-bdc4dd16c6fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1843d1fa62bd4ee2b05c46b7000f36b2', 'request_id': 'req-e533302f-cc52-4cac-98f7-bdc4dd16c6fd'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:20.041211 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e533302f-cc52-4cac-98f7-bdc4dd16c6fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1843d1fa62bd4ee2b05c46b7000f36b2', 'request_id': 'req-e533302f-cc52-4cac-98f7-bdc4dd16c6fd'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:20.048921 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e533302f-cc52-4cac-98f7-bdc4dd16c6fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1843d1fa62bd4ee2b05c46b7000f36b2', 'request_id': 'req-e533302f-cc52-4cac-98f7-bdc4dd16c6fd'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:20.060520 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e533302f-cc52-4cac-98f7-bdc4dd16c6fd admin admin] The token cache is being invalidate because project 1843d1fa62bd4ee2b05c46b7000f36b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:20.060868 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e533302f-cc52-4cac-98f7-bdc4dd16c6fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:20.062439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e533302f-cc52-4cac-98f7-bdc4dd16c6fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:20.064026 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 911/1817] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:19 2022] DELETE /identity/v3/projects/1843d1fa62bd4ee2b05c46b7000f36b2 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:20.067906 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5087d-8d49-4a82-95d7-8c673325ad25 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:20.069462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-eee5087d-8d49-4a82-95d7-8c673325ad25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:20.151332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5087d-8d49-4a82-95d7-8c673325ad25 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:20.158241 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c5883-019c-4455-a52d-129c34595af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:20.159902 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c5883-019c-4455-a52d-129c34595af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:20.160543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c5883-019c-4455-a52d-129c34595af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:20.161018 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c5883-019c-4455-a52d-129c34595af3 admin admin] PATH_INFO: `/v3/users/8f3c2acd8c9049299186275db8e727ac` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:20.162890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c5883-019c-4455-a52d-129c34595af3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f3c2acd8c9049299186275db8e727ac)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:20.168080 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c5883-019c-4455-a52d-129c34595af3 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:20.341376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533302f-cc52-4cac-98f7-bdc4dd16c6fd admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:20.342593 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-e533302f-cc52-4cac-98f7-bdc4dd16c6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:20.424515 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533302f-cc52-4cac-98f7-bdc4dd16c6fd admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:20.437360 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffc2b9-71a0-474c-8bbf-0c4405e6fb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:20.438912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffc2b9-71a0-474c-8bbf-0c4405e6fb51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:20.440059 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffc2b9-71a0-474c-8bbf-0c4405e6fb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:20.441420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffc2b9-71a0-474c-8bbf-0c4405e6fb51 admin admin] PATH_INFO: `/v3/projects/85cf60ccc83946d9b53672f9667dfe8c` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:20.443918 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffc2b9-71a0-474c-8bbf-0c4405e6fb51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85cf60ccc83946d9b53672f9667dfe8c)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:20.448390 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffc2b9-71a0-474c-8bbf-0c4405e6fb51 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:20.458517 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-5b4c5883-019c-4455-a52d-129c34595af3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f3c2acd8c9049299186275db8e727ac'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:20.459277 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-5b4c5883-019c-4455-a52d-129c34595af3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f3c2acd8c9049299186275db8e727ac'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:20.480813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-5b4c5883-019c-4455-a52d-129c34595af3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f3c2acd8c9049299186275db8e727ac'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:20.482620 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-5b4c5883-019c-4455-a52d-129c34595af3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f3c2acd8c9049299186275db8e727ac'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:20.496060 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-5b4c5883-019c-4455-a52d-129c34595af3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f3c2acd8c9049299186275db8e727ac'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:20.595248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 907/1818] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:20 2022] DELETE /identity/v3/users/8f3c2acd8c9049299186275db8e727ac => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:20.658144 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2bffc2b9-71a0-474c-8bbf-0c4405e6fb51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85cf60ccc83946d9b53672f9667dfe8c', 'request_id': 'req-2bffc2b9-71a0-474c-8bbf-0c4405e6fb51'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:20.742747 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2bffc2b9-71a0-474c-8bbf-0c4405e6fb51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85cf60ccc83946d9b53672f9667dfe8c', 'request_id': 'req-2bffc2b9-71a0-474c-8bbf-0c4405e6fb51'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:20.749075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2bffc2b9-71a0-474c-8bbf-0c4405e6fb51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85cf60ccc83946d9b53672f9667dfe8c', 'request_id': 'req-2bffc2b9-71a0-474c-8bbf-0c4405e6fb51'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:20.760779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2bffc2b9-71a0-474c-8bbf-0c4405e6fb51 admin admin] The token cache is being invalidate because project 85cf60ccc83946d9b53672f9667dfe8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:20.761067 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2bffc2b9-71a0-474c-8bbf-0c4405e6fb51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:20.761733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2bffc2b9-71a0-474c-8bbf-0c4405e6fb51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:20.766333 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 912/1819] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:20 2022] DELETE /identity/v3/projects/85cf60ccc83946d9b53672f9667dfe8c => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:20.767578 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c5883-019c-4455-a52d-129c34595af3 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:20.768991 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-5b4c5883-019c-4455-a52d-129c34595af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:20.790629 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffc2b9-71a0-474c-8bbf-0c4405e6fb51 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:20.792625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-2bffc2b9-71a0-474c-8bbf-0c4405e6fb51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:20.856841 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c5883-019c-4455-a52d-129c34595af3 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:20.874654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffc2b9-71a0-474c-8bbf-0c4405e6fb51 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:20.879289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a74e9-f458-478f-8c68-43290b5ef776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:20.880138 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a74e9-f458-478f-8c68-43290b5ef776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:20.880427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a74e9-f458-478f-8c68-43290b5ef776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:20.880702 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a74e9-f458-478f-8c68-43290b5ef776 admin admin] PATH_INFO: `/v3/projects/b0ed9b7ec483461698d3a620216fa426` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:20.882491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a74e9-f458-478f-8c68-43290b5ef776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0ed9b7ec483461698d3a620216fa426)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:20.885073 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a74e9-f458-478f-8c68-43290b5ef776 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:20.915234 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70f644-bd81-4f9d-8354-913ccbd70dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:20.916157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70f644-bd81-4f9d-8354-913ccbd70dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:20.916434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70f644-bd81-4f9d-8354-913ccbd70dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:20.916706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70f644-bd81-4f9d-8354-913ccbd70dce admin admin] PATH_INFO: `/v3/users/30c259d145e143cfad915d6e3b6f17dc` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:20.917919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70f644-bd81-4f9d-8354-913ccbd70dce admin admin] RBAC: Authorizing `identity:delete_user(user_id=30c259d145e143cfad915d6e3b6f17dc)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:20.919888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70f644-bd81-4f9d-8354-913ccbd70dce admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:21.013344 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-363a74e9-f458-478f-8c68-43290b5ef776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0ed9b7ec483461698d3a620216fa426', 'request_id': 'req-363a74e9-f458-478f-8c68-43290b5ef776'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:21.081199 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-363a74e9-f458-478f-8c68-43290b5ef776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0ed9b7ec483461698d3a620216fa426', 'request_id': 'req-363a74e9-f458-478f-8c68-43290b5ef776'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:21.096257 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-363a74e9-f458-478f-8c68-43290b5ef776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0ed9b7ec483461698d3a620216fa426', 'request_id': 'req-363a74e9-f458-478f-8c68-43290b5ef776'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:21.108081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-363a74e9-f458-478f-8c68-43290b5ef776 admin admin] The token cache is being invalidate because project b0ed9b7ec483461698d3a620216fa426 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:21.108282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-363a74e9-f458-478f-8c68-43290b5ef776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:21.108946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-363a74e9-f458-478f-8c68-43290b5ef776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:21.110555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 913/1820] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:20 2022] DELETE /identity/v3/projects/b0ed9b7ec483461698d3a620216fa426 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:21.115192 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970895c-4df1-4a5b-8f8b-66050e6b78cf None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:21.115501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970895c-4df1-4a5b-8f8b-66050e6b78cf None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:21.121050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970895c-4df1-4a5b-8f8b-66050e6b78cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:21.134288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-3970895c-4df1-4a5b-8f8b-66050e6b78cf None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:21.135498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3970895c-4df1-4a5b-8f8b-66050e6b78cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:21.147144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-9b70f644-bd81-4f9d-8354-913ccbd70dce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30c259d145e143cfad915d6e3b6f17dc'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:21.149490 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-9b70f644-bd81-4f9d-8354-913ccbd70dce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30c259d145e143cfad915d6e3b6f17dc'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:21.162728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-9b70f644-bd81-4f9d-8354-913ccbd70dce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30c259d145e143cfad915d6e3b6f17dc'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:21.163433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-9b70f644-bd81-4f9d-8354-913ccbd70dce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30c259d145e143cfad915d6e3b6f17dc'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:21.170716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-9b70f644-bd81-4f9d-8354-913ccbd70dce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30c259d145e143cfad915d6e3b6f17dc'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:21.195980 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 914/1821] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:21.203535 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dded3ef-3503-45be-8622-c0f780e17327 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:21.203823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dded3ef-3503-45be-8622-c0f780e17327 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:21.207632 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dded3ef-3503-45be-8622-c0f780e17327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:21.221259 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-5dded3ef-3503-45be-8622-c0f780e17327 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:21.222486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-5dded3ef-3503-45be-8622-c0f780e17327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:21.231520 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 915/1822] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:21.232208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dded3ef-3503-45be-8622-c0f780e17327 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:21.233578 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-5dded3ef-3503-45be-8622-c0f780e17327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:21.239947 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 908/1823] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:20 2022] DELETE /identity/v3/users/30c259d145e143cfad915d6e3b6f17dc => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:21.244308 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70f644-bd81-4f9d-8354-913ccbd70dce admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:21.246317 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dded3ef-3503-45be-8622-c0f780e17327 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:21.317875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70f644-bd81-4f9d-8354-913ccbd70dce admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:21.324167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feafac9-be7b-4e1e-a8fd-144a4b1d5f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:21.324854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feafac9-be7b-4e1e-a8fd-144a4b1d5f2e admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:21.325185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feafac9-be7b-4e1e-a8fd-144a4b1d5f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:21.325482 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feafac9-be7b-4e1e-a8fd-144a4b1d5f2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:21.326385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feafac9-be7b-4e1e-a8fd-144a4b1d5f2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:17:21.326752 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feafac9-be7b-4e1e-a8fd-144a4b1d5f2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:21.328402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feafac9-be7b-4e1e-a8fd-144a4b1d5f2e admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:21.329304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba81c9-da20-467b-87a7-96daf791db29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:21.330602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba81c9-da20-467b-87a7-96daf791db29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:21.330884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba81c9-da20-467b-87a7-96daf791db29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:21.331472 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba81c9-da20-467b-87a7-96daf791db29 admin admin] PATH_INFO: `/v3/projects/e83ad63d060e4abe95356d1efb54ad17` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:21.332406 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba81c9-da20-467b-87a7-96daf791db29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e83ad63d060e4abe95356d1efb54ad17)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:21.334790 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba81c9-da20-467b-87a7-96daf791db29 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:21.341834 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 909/1824] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:17:21.346457 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feafac9-be7b-4e1e-a8fd-144a4b1d5f2e admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:21.356795 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feafac9-be7b-4e1e-a8fd-144a4b1d5f2e admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:21.362422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95f6d8-3acf-48a0-b1a5-72cae0dbc46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:21.363178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95f6d8-3acf-48a0-b1a5-72cae0dbc46e admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:21.363561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95f6d8-3acf-48a0-b1a5-72cae0dbc46e admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:21.363903 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95f6d8-3acf-48a0-b1a5-72cae0dbc46e admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:21.364666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95f6d8-3acf-48a0-b1a5-72cae0dbc46e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:21.366660 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95f6d8-3acf-48a0-b1a5-72cae0dbc46e admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:21.454218 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 910/1825] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:21 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:17:21.461353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95f6d8-3acf-48a0-b1a5-72cae0dbc46e admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:21.480442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b7ba81c9-da20-467b-87a7-96daf791db29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e83ad63d060e4abe95356d1efb54ad17', 'request_id': 'req-b7ba81c9-da20-467b-87a7-96daf791db29'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:21.527787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95f6d8-3acf-48a0-b1a5-72cae0dbc46e admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:21.532565 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42250466-1929-4b29-93da-cbb0032cd488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:21.533240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42250466-1929-4b29-93da-cbb0032cd488 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:21.533529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42250466-1929-4b29-93da-cbb0032cd488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:21.533811 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42250466-1929-4b29-93da-cbb0032cd488 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:21.534544 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42250466-1929-4b29-93da-cbb0032cd488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:21.536487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42250466-1929-4b29-93da-cbb0032cd488 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:21.555388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b7ba81c9-da20-467b-87a7-96daf791db29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e83ad63d060e4abe95356d1efb54ad17', 'request_id': 'req-b7ba81c9-da20-467b-87a7-96daf791db29'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:21.561375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b7ba81c9-da20-467b-87a7-96daf791db29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e83ad63d060e4abe95356d1efb54ad17', 'request_id': 'req-b7ba81c9-da20-467b-87a7-96daf791db29'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:21.572420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b7ba81c9-da20-467b-87a7-96daf791db29 admin admin] The token cache is being invalidate because project e83ad63d060e4abe95356d1efb54ad17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:21.572710 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b7ba81c9-da20-467b-87a7-96daf791db29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:21.573472 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b7ba81c9-da20-467b-87a7-96daf791db29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:21.575322 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 916/1826] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:21 2022] DELETE /identity/v3/projects/e83ad63d060e4abe95356d1efb54ad17 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:21.581397 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50493b43-7edf-4247-9705-a0a34d2de118 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:21.581687 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50493b43-7edf-4247-9705-a0a34d2de118 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:21.581968 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50493b43-7edf-4247-9705-a0a34d2de118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:21.600113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-50493b43-7edf-4247-9705-a0a34d2de118 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:21.601317 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-50493b43-7edf-4247-9705-a0a34d2de118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:21.610389 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 917/1827] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:21.618039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a4e50-ebb7-4758-8997-8ddcd9468523 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:21.618368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a4e50-ebb7-4758-8997-8ddcd9468523 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:21.618653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a4e50-ebb7-4758-8997-8ddcd9468523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:21.642766 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-597a4e50-ebb7-4758-8997-8ddcd9468523 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:21.644313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-597a4e50-ebb7-4758-8997-8ddcd9468523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:21.655962 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 918/1828] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:21.666689 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a4e50-ebb7-4758-8997-8ddcd9468523 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:21.673039 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 911/1829] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:21 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:17:21.674115 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42250466-1929-4b29-93da-cbb0032cd488 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:21.678209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-42250466-1929-4b29-93da-cbb0032cd488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:21.682979 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a4e50-ebb7-4758-8997-8ddcd9468523 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:21.695610 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe047604-6f56-4ef4-b918-08dfee31c59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:21.696452 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe047604-6f56-4ef4-b918-08dfee31c59f admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:21.696812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe047604-6f56-4ef4-b918-08dfee31c59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:21.697434 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe047604-6f56-4ef4-b918-08dfee31c59f admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:21.702564 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe047604-6f56-4ef4-b918-08dfee31c59f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:17:21.703012 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe047604-6f56-4ef4-b918-08dfee31c59f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:21.708375 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42250466-1929-4b29-93da-cbb0032cd488 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:21.708715 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe047604-6f56-4ef4-b918-08dfee31c59f admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:21.717951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1322aa2-2e09-4bb8-a62c-e4c78e34096d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:21.718761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1322aa2-2e09-4bb8-a62c-e4c78e34096d admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:21.719053 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1322aa2-2e09-4bb8-a62c-e4c78e34096d admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:21.719339 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1322aa2-2e09-4bb8-a62c-e4c78e34096d admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:21.720132 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1322aa2-2e09-4bb8-a62c-e4c78e34096d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:21.721729 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1322aa2-2e09-4bb8-a62c-e4c78e34096d admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:21.731167 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 919/1830] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 20:17:21.732012 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe047604-6f56-4ef4-b918-08dfee31c59f admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:21.743317 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-fe047604-6f56-4ef4-b918-08dfee31c59f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:21.754643 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 912/1831] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:21.756245 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1322aa2-2e09-4bb8-a62c-e4c78e34096d admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:21.777882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe047604-6f56-4ef4-b918-08dfee31c59f admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:21.778062 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1322aa2-2e09-4bb8-a62c-e4c78e34096d admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:21.778768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79f605-f200-4c22-8c84-8383827c2536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:21.779645 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79f605-f200-4c22-8c84-8383827c2536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:21.779930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79f605-f200-4c22-8c84-8383827c2536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:21.780210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79f605-f200-4c22-8c84-8383827c2536 admin admin] PATH_INFO: `/v3/projects/ca49683347ca43abb4db836e47e8fe85` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:21.782032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79f605-f200-4c22-8c84-8383827c2536 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca49683347ca43abb4db836e47e8fe85)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:21.783647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacf427-3889-4924-93aa-6a1ccb209f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:21.783984 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79f605-f200-4c22-8c84-8383827c2536 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:21.784988 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacf427-3889-4924-93aa-6a1ccb209f79 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:21.785568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacf427-3889-4924-93aa-6a1ccb209f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:21.786331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacf427-3889-4924-93aa-6a1ccb209f79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:21.787614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacf427-3889-4924-93aa-6a1ccb209f79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:21.791208 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacf427-3889-4924-93aa-6a1ccb209f79 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:21.881283 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 913/1832] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:21 2022] POST /identity/v3/projects => generated 369 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:21.881763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacf427-3889-4924-93aa-6a1ccb209f79 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:21.926003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7b79f605-f200-4c22-8c84-8383827c2536 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca49683347ca43abb4db836e47e8fe85', 'request_id': 'req-7b79f605-f200-4c22-8c84-8383827c2536'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:21.969411 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacf427-3889-4924-93aa-6a1ccb209f79 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:22.007295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7b79f605-f200-4c22-8c84-8383827c2536 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca49683347ca43abb4db836e47e8fe85', 'request_id': 'req-7b79f605-f200-4c22-8c84-8383827c2536'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:22.015497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7b79f605-f200-4c22-8c84-8383827c2536 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca49683347ca43abb4db836e47e8fe85', 'request_id': 'req-7b79f605-f200-4c22-8c84-8383827c2536'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:22.028533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7b79f605-f200-4c22-8c84-8383827c2536 admin admin] The token cache is being invalidate because project ca49683347ca43abb4db836e47e8fe85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:22.028533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7b79f605-f200-4c22-8c84-8383827c2536 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:22.030793 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7b79f605-f200-4c22-8c84-8383827c2536 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:22.037447 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 920/1833] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:21 2022] DELETE /identity/v3/projects/ca49683347ca43abb4db836e47e8fe85 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:22.038152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79f605-f200-4c22-8c84-8383827c2536 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:22.039741 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7b79f605-f200-4c22-8c84-8383827c2536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:22.062224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7afdc76-0cea-4345-b6bb-e3746aaee651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:22.064005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afdc76-0cea-4345-b6bb-e3746aaee651 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:22.065795 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afdc76-0cea-4345-b6bb-e3746aaee651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:22.065916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afdc76-0cea-4345-b6bb-e3746aaee651 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:22.069004 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7afdc76-0cea-4345-b6bb-e3746aaee651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:22.074026 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7afdc76-0cea-4345-b6bb-e3746aaee651 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:22.091432 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 914/1834] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:22.093177 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0eef9-665d-4206-a268-3611af51073d None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:22.093482 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0eef9-665d-4206-a268-3611af51073d None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:22.093767 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0eef9-665d-4206-a268-3611af51073d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:22.116845 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-12c0eef9-665d-4206-a268-3611af51073d None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:22.125750 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-12c0eef9-665d-4206-a268-3611af51073d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:22.128511 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 915/1835] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:22.129592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0eef9-665d-4206-a268-3611af51073d None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:22.131129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-12c0eef9-665d-4206-a268-3611af51073d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:22.147813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0eef9-665d-4206-a268-3611af51073d None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:22.151954 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79f605-f200-4c22-8c84-8383827c2536 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:22.154489 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5b906-45c3-4e3e-bfcc-bcfb8a4e6589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:22.155904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5b906-45c3-4e3e-bfcc-bcfb8a4e6589 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:22.156260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5b906-45c3-4e3e-bfcc-bcfb8a4e6589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:22.156692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5b906-45c3-4e3e-bfcc-bcfb8a4e6589 admin admin] PATH_INFO: `/v3/projects/8963f6d764774712a00076158990daa1/users/e1a793e23b934ddb83ea6573fa1eb81a/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:22.162770 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb794b2-7c7d-4eec-8a8b-3536610e888d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:22.171268 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb794b2-7c7d-4eec-8a8b-3536610e888d admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:22.174275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb794b2-7c7d-4eec-8a8b-3536610e888d admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:22.175696 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb794b2-7c7d-4eec-8a8b-3536610e888d admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:22.176849 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb794b2-7c7d-4eec-8a8b-3536610e888d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:22.177915 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5b906-45c3-4e3e-bfcc-bcfb8a4e6589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8963f6d764774712a00076158990daa1, user_id=e1a793e23b934ddb83ea6573fa1eb81a, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:22.180861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5b906-45c3-4e3e-bfcc-bcfb8a4e6589 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:22.182498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb794b2-7c7d-4eec-8a8b-3536610e888d admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:22.270702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 916/1836] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:22 2022] PUT /identity/v3/projects/8963f6d764774712a00076158990daa1/users/e1a793e23b934ddb83ea6573fa1eb81a/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:22.279953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9343a2-3dc4-4ed1-ab4f-bbdf9eec77af None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:22.279953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9343a2-3dc4-4ed1-ab4f-bbdf9eec77af None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:22.279953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9343a2-3dc4-4ed1-ab4f-bbdf9eec77af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:22.297325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 921/1837] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:22 2022] POST /identity/v3/users => generated 285 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:22.297325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa750be7-148d-4e1f-9ab0-abf8406201f9 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:22.297543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa750be7-148d-4e1f-9ab0-abf8406201f9 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:22.297809 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa750be7-148d-4e1f-9ab0-abf8406201f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:22.310167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-4e9343a2-3dc4-4ed1-ab4f-bbdf9eec77af None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:22.311560 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-4e9343a2-3dc4-4ed1-ab4f-bbdf9eec77af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:22.337086 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-fa750be7-148d-4e1f-9ab0-abf8406201f9 None None] MFA Rules not processed for user `e1a793e23b934ddb83ea6573fa1eb81a`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:22.338716 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-fa750be7-148d-4e1f-9ab0-abf8406201f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:22.398546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 917/1838] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:22.399737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9343a2-3dc4-4ed1-ab4f-bbdf9eec77af None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:22.409825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9343a2-3dc4-4ed1-ab4f-bbdf9eec77af None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:22.415992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91ed63-2449-429a-bce0-7ca56cf2c4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:22.416919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91ed63-2449-429a-bce0-7ca56cf2c4e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:22.417376 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91ed63-2449-429a-bce0-7ca56cf2c4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:22.417667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91ed63-2449-429a-bce0-7ca56cf2c4e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:22.418513 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91ed63-2449-429a-bce0-7ca56cf2c4e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:22.420256 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91ed63-2449-429a-bce0-7ca56cf2c4e2 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:22.437329 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 918/1839] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:22.437329 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91ed63-2449-429a-bce0-7ca56cf2c4e2 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:22.460453 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91ed63-2449-429a-bce0-7ca56cf2c4e2 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:22.472303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e7008-eaed-45c8-bc60-cac302f5ca44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:22.473555 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e7008-eaed-45c8-bc60-cac302f5ca44 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:22.473853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e7008-eaed-45c8-bc60-cac302f5ca44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:22.474153 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e7008-eaed-45c8-bc60-cac302f5ca44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:22.474921 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e7008-eaed-45c8-bc60-cac302f5ca44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:17:22.475217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e7008-eaed-45c8-bc60-cac302f5ca44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:22.477224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e7008-eaed-45c8-bc60-cac302f5ca44 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:22.492851 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 922/1840] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:22.492851 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa750be7-148d-4e1f-9ab0-abf8406201f9 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:22.500302 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 919/1841] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:17:22.501187 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e7008-eaed-45c8-bc60-cac302f5ca44 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:22.511240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa750be7-148d-4e1f-9ab0-abf8406201f9 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:22.512794 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e7008-eaed-45c8-bc60-cac302f5ca44 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:22.518244 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dac524-7ed4-49ce-ba19-945c6cfc4e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:22.519414 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dac524-7ed4-49ce-ba19-945c6cfc4e00 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:22.519798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dac524-7ed4-49ce-ba19-945c6cfc4e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:22.520422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6146b3b-881c-4d88-a018-8b46ef203899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:22.520857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dac524-7ed4-49ce-ba19-945c6cfc4e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:22.521226 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6146b3b-881c-4d88-a018-8b46ef203899 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:22.521383 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6146b3b-881c-4d88-a018-8b46ef203899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:22.521662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6146b3b-881c-4d88-a018-8b46ef203899 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:22.521985 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dac524-7ed4-49ce-ba19-945c6cfc4e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:22.523327 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6146b3b-881c-4d88-a018-8b46ef203899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:22.523988 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dac524-7ed4-49ce-ba19-945c6cfc4e00 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:22.526559 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6146b3b-881c-4d88-a018-8b46ef203899 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:22.543468 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 923/1842] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:22.544392 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dac524-7ed4-49ce-ba19-945c6cfc4e00 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:22.559186 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dac524-7ed4-49ce-ba19-945c6cfc4e00 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:22.569835 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2318984-e99f-4953-b13c-173065df9cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:22.569835 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2318984-e99f-4953-b13c-173065df9cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:22.569835 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2318984-e99f-4953-b13c-173065df9cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:22.569835 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2318984-e99f-4953-b13c-173065df9cc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:22.571020 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2318984-e99f-4953-b13c-173065df9cc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:22.571020 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2318984-e99f-4953-b13c-173065df9cc0 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:22.649484 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 920/1843] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:22 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:22.650535 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6146b3b-881c-4d88-a018-8b46ef203899 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:22.672971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 924/1844] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:22 2022] POST /identity/v3/projects => generated 381 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:22.673959 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2318984-e99f-4953-b13c-173065df9cc0 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:22.730765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6146b3b-881c-4d88-a018-8b46ef203899 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:22.765762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2318984-e99f-4953-b13c-173065df9cc0 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:22.772065 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdee8d-f921-4c96-a38e-6edae4b49165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:22.772823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdee8d-f921-4c96-a38e-6edae4b49165 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:22.773092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdee8d-f921-4c96-a38e-6edae4b49165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:22.773748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdee8d-f921-4c96-a38e-6edae4b49165 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:22.774548 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdee8d-f921-4c96-a38e-6edae4b49165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:22.777396 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdee8d-f921-4c96-a38e-6edae4b49165 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:22.804100 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b5949-b0a2-4cac-ac69-d90dac903daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:22.805142 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b5949-b0a2-4cac-ac69-d90dac903daf admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:22.805493 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b5949-b0a2-4cac-ac69-d90dac903daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:22.805800 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b5949-b0a2-4cac-ac69-d90dac903daf admin admin] PATH_INFO: `/v3/projects/8c0ca727fd7f4a86b103933126caae5e/users/cc740d5c78b348b8a8497a78560a909c/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:22.829049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b5949-b0a2-4cac-ac69-d90dac903daf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c0ca727fd7f4a86b103933126caae5e, user_id=cc740d5c78b348b8a8497a78560a909c, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:22.831833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b5949-b0a2-4cac-ac69-d90dac903daf admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:22.905220 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 925/1845] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:22 2022] POST /identity/v3/users => generated 295 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:22.906501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdee8d-f921-4c96-a38e-6edae4b49165 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:22.919071 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdee8d-f921-4c96-a38e-6edae4b49165 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:22.925174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f1f70-7503-40b9-aeb3-e308f86ea1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:22.925826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f1f70-7503-40b9-aeb3-e308f86ea1d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:22.926219 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f1f70-7503-40b9-aeb3-e308f86ea1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:22.926501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f1f70-7503-40b9-aeb3-e308f86ea1d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:22.927531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f1f70-7503-40b9-aeb3-e308f86ea1d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:22.929717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f1f70-7503-40b9-aeb3-e308f86ea1d6 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:22.956262 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 921/1846] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:22 2022] PUT /identity/v3/projects/8c0ca727fd7f4a86b103933126caae5e/users/cc740d5c78b348b8a8497a78560a909c/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:22.957099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b5949-b0a2-4cac-ac69-d90dac903daf admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:23.019337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b5949-b0a2-4cac-ac69-d90dac903daf admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:23.032514 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 926/1847] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:22 2022] POST /identity/v3/users => generated 291 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:23.033811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8973a-8042-4cce-b2b4-a79e217ebe98 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:23.034118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8973a-8042-4cce-b2b4-a79e217ebe98 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:23.035261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8973a-8042-4cce-b2b4-a79e217ebe98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:23.038393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a96ca-06b5-4e6a-97ce-cbf7959dbc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:23.039082 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a96ca-06b5-4e6a-97ce-cbf7959dbc19 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:23.039364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a96ca-06b5-4e6a-97ce-cbf7959dbc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:23.039636 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a96ca-06b5-4e6a-97ce-cbf7959dbc19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:23.041588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a96ca-06b5-4e6a-97ce-cbf7959dbc19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:23.043240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a96ca-06b5-4e6a-97ce-cbf7959dbc19 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:23.063302 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 922/1848] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:23.064109 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a96ca-06b5-4e6a-97ce-cbf7959dbc19 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:23.076875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a96ca-06b5-4e6a-97ce-cbf7959dbc19 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:23.077688 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-1fe8973a-8042-4cce-b2b4-a79e217ebe98 None None] MFA Rules not processed for user `cc740d5c78b348b8a8497a78560a909c`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:23.079032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-1fe8973a-8042-4cce-b2b4-a79e217ebe98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:23.082961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d5dc0-834a-4fc8-9abf-d4d67cac4b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:23.083721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d5dc0-834a-4fc8-9abf-d4d67cac4b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:23.084234 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d5dc0-834a-4fc8-9abf-d4d67cac4b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:23.084589 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d5dc0-834a-4fc8-9abf-d4d67cac4b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:23.085912 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d5dc0-834a-4fc8-9abf-d4d67cac4b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:23.088624 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d5dc0-834a-4fc8-9abf-d4d67cac4b57 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:23.104428 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 923/1849] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:23.104428 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d5dc0-834a-4fc8-9abf-d4d67cac4b57 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:23.114974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d5dc0-834a-4fc8-9abf-d4d67cac4b57 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:23.120587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d84fa3-c63b-4185-b396-1c1493001b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:23.121447 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d84fa3-c63b-4185-b396-1c1493001b54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:23.121734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d84fa3-c63b-4185-b396-1c1493001b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:23.122047 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d84fa3-c63b-4185-b396-1c1493001b54 admin admin] PATH_INFO: `/v3/projects/41cb7b51ac4a449e81af84abff566bcd/users/194ce1f9b8b441f4bbbe8d51bb800ec6/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:23.139857 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d84fa3-c63b-4185-b396-1c1493001b54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41cb7b51ac4a449e81af84abff566bcd, user_id=194ce1f9b8b441f4bbbe8d51bb800ec6, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:23.142557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d84fa3-c63b-4185-b396-1c1493001b54 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:23.161473 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 927/1850] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:23.162401 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8973a-8042-4cce-b2b4-a79e217ebe98 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:23.175315 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8973a-8042-4cce-b2b4-a79e217ebe98 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:23.180380 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6789c-38f8-45d0-9c8d-e53cb1a307bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:23.181046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6789c-38f8-45d0-9c8d-e53cb1a307bd admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:23.181358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6789c-38f8-45d0-9c8d-e53cb1a307bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:23.181632 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6789c-38f8-45d0-9c8d-e53cb1a307bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:23.182402 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6789c-38f8-45d0-9c8d-e53cb1a307bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:23.184163 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6789c-38f8-45d0-9c8d-e53cb1a307bd admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:23.194139 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 928/1851] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:23.194930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6789c-38f8-45d0-9c8d-e53cb1a307bd admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:23.196575 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a6e6789c-38f8-45d0-9c8d-e53cb1a307bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:23.224055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 924/1852] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:23 2022] PUT /identity/v3/projects/41cb7b51ac4a449e81af84abff566bcd/users/194ce1f9b8b441f4bbbe8d51bb800ec6/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:23.224897 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d84fa3-c63b-4185-b396-1c1493001b54 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:23.254400 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6789c-38f8-45d0-9c8d-e53cb1a307bd admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:23.285799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d84fa3-c63b-4185-b396-1c1493001b54 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:23.291414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7c2b1-3509-4566-8eca-0de4b23f5528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:23.292477 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7c2b1-3509-4566-8eca-0de4b23f5528 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:23.292769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7c2b1-3509-4566-8eca-0de4b23f5528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:23.293065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7c2b1-3509-4566-8eca-0de4b23f5528 admin admin] PATH_INFO: `/v3/projects/b367870b55cf42b19eec3b14a9bfc8aa/users/156ae7d75dd64d82b30214f345f63513/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:23.305583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69503b75-bc11-4506-a4b8-05670760fda9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67a252cde71a455fb1dc4bf9c47c0490', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:23.306313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69503b75-bc11-4506-a4b8-05670760fda9 service glance] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:23.306838 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69503b75-bc11-4506-a4b8-05670760fda9 service glance] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:23.306964 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69503b75-bc11-4506-a4b8-05670760fda9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:23.328445 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7c2b1-3509-4566-8eca-0de4b23f5528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b367870b55cf42b19eec3b14a9bfc8aa, user_id=156ae7d75dd64d82b30214f345f63513, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:23.328445 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7c2b1-3509-4566-8eca-0de4b23f5528 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:23.370048 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69503b75-bc11-4506-a4b8-05670760fda9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:23.371547 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69503b75-bc11-4506-a4b8-05670760fda9 service glance] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:23.380143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 929/1853] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:23 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:17:23.381000 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69503b75-bc11-4506-a4b8-05670760fda9 service glance] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:23.391708 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69503b75-bc11-4506-a4b8-05670760fda9 service glance] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:23.396465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5fbf3-0e91-49d7-96de-92797fff40f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:23.397122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5fbf3-0e91-49d7-96de-92797fff40f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:23.397409 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5fbf3-0e91-49d7-96de-92797fff40f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:23.397683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5fbf3-0e91-49d7-96de-92797fff40f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:23.398607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5fbf3-0e91-49d7-96de-92797fff40f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:23.406645 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5fbf3-0e91-49d7-96de-92797fff40f7 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:23.412527 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 930/1854] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:23.419858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5fbf3-0e91-49d7-96de-92797fff40f7 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:23.432654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5fbf3-0e91-49d7-96de-92797fff40f7 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:23.438222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d5c93-e87c-49c6-9317-065983be731b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:23.439183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d5c93-e87c-49c6-9317-065983be731b admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:23.439529 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d5c93-e87c-49c6-9317-065983be731b admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:23.439870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d5c93-e87c-49c6-9317-065983be731b admin admin] PATH_INFO: `/v3/projects/41cb7b51ac4a449e81af84abff566bcd/users/194ce1f9b8b441f4bbbe8d51bb800ec6/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:23.442925 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d5c93-e87c-49c6-9317-065983be731b admin admin] RBAC: Authorizing `identity:create_grant(project_id=41cb7b51ac4a449e81af84abff566bcd, user_id=194ce1f9b8b441f4bbbe8d51bb800ec6, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:23.445741 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d5c93-e87c-49c6-9317-065983be731b admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:23.457996 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 931/1855] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:23 2022] PUT /identity/v3/projects/41cb7b51ac4a449e81af84abff566bcd/users/194ce1f9b8b441f4bbbe8d51bb800ec6/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:23.468080 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 925/1856] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:23 2022] PUT /identity/v3/projects/b367870b55cf42b19eec3b14a9bfc8aa/users/156ae7d75dd64d82b30214f345f63513/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:23.472661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd909e5-e509-4c14-8c55-4f3cf5818b5c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:23.472970 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd909e5-e509-4c14-8c55-4f3cf5818b5c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:23.473249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd909e5-e509-4c14-8c55-4f3cf5818b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:23.478880 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4dc212-4646-49f5-969e-777ce273ebaf None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:23.479364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4dc212-4646-49f5-969e-777ce273ebaf None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:23.479848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4dc212-4646-49f5-969e-777ce273ebaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:23.505282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-5cd909e5-e509-4c14-8c55-4f3cf5818b5c None None] MFA Rules not processed for user `194ce1f9b8b441f4bbbe8d51bb800ec6`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:23.507027 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-5cd909e5-e509-4c14-8c55-4f3cf5818b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:23.517188 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-6b4dc212-4646-49f5-969e-777ce273ebaf None None] MFA Rules not processed for user `156ae7d75dd64d82b30214f345f63513`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:23.518878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-6b4dc212-4646-49f5-969e-777ce273ebaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:23.618028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 932/1857] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:23.625880 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd909e5-e509-4c14-8c55-4f3cf5818b5c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:23.627548 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-5cd909e5-e509-4c14-8c55-4f3cf5818b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:23.628325 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 926/1858] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:23.635938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4dc212-4646-49f5-969e-777ce273ebaf None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:23.693885 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd909e5-e509-4c14-8c55-4f3cf5818b5c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:23.698997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966157f2-00b1-43a5-b6d4-021b57539588 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:23.699646 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966157f2-00b1-43a5-b6d4-021b57539588 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:23.700009 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966157f2-00b1-43a5-b6d4-021b57539588 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:23.700256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966157f2-00b1-43a5-b6d4-021b57539588 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:23.702942 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4dc212-4646-49f5-969e-777ce273ebaf None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:23.708165 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95542684-7e80-4828-96c0-d9cb7d692b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:23.708931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95542684-7e80-4828-96c0-d9cb7d692b6e admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:23.709310 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95542684-7e80-4828-96c0-d9cb7d692b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:23.709653 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95542684-7e80-4828-96c0-d9cb7d692b6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:23.710608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95542684-7e80-4828-96c0-d9cb7d692b6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:23.713148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95542684-7e80-4828-96c0-d9cb7d692b6e admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:23.757667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966157f2-00b1-43a5-b6d4-021b57539588 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:23.759131 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966157f2-00b1-43a5-b6d4-021b57539588 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:23.768981 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 933/1859] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:17:23.815248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 927/1860] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:23 2022] POST /identity/v3/projects => generated 383 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:23.819514 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966157f2-00b1-43a5-b6d4-021b57539588 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:23.830706 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966157f2-00b1-43a5-b6d4-021b57539588 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:23.888088 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36d26a-ef70-4c6e-801d-5bbaf7f5c4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:23.888751 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36d26a-ef70-4c6e-801d-5bbaf7f5c4e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:23.889093 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36d26a-ef70-4c6e-801d-5bbaf7f5c4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:23.889373 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36d26a-ef70-4c6e-801d-5bbaf7f5c4e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:23.890065 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36d26a-ef70-4c6e-801d-5bbaf7f5c4e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:23.892048 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36d26a-ef70-4c6e-801d-5bbaf7f5c4e4 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:23.983667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 934/1861] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:23 2022] POST /identity/v3/users => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:23.987922 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95542684-7e80-4828-96c0-d9cb7d692b6e admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:24.000059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95542684-7e80-4828-96c0-d9cb7d692b6e admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:24.005113 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99907fc5-1c78-4f71-966e-ecbef9b5f182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:24.005929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99907fc5-1c78-4f71-966e-ecbef9b5f182 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:24.006196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99907fc5-1c78-4f71-966e-ecbef9b5f182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:24.006472 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99907fc5-1c78-4f71-966e-ecbef9b5f182 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:24.007219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99907fc5-1c78-4f71-966e-ecbef9b5f182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:24.008898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99907fc5-1c78-4f71-966e-ecbef9b5f182 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:24.020509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 928/1862] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:24.024923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36d26a-ef70-4c6e-801d-5bbaf7f5c4e4 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:24.035047 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36d26a-ef70-4c6e-801d-5bbaf7f5c4e4 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:24.047832 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a00b4d-6571-47f8-9bd3-ffdc297fba57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:24.049229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a00b4d-6571-47f8-9bd3-ffdc297fba57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:24.049517 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a00b4d-6571-47f8-9bd3-ffdc297fba57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:24.049997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a00b4d-6571-47f8-9bd3-ffdc297fba57 admin admin] PATH_INFO: `/v3/projects/e789f0ff432a4b65b56ae384889c2fa0/users/7550ea5a5d264d029ede1ff5f3e13f97/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:24.070245 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a00b4d-6571-47f8-9bd3-ffdc297fba57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e789f0ff432a4b65b56ae384889c2fa0, user_id=7550ea5a5d264d029ede1ff5f3e13f97, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:24.074921 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a00b4d-6571-47f8-9bd3-ffdc297fba57 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:24.198698 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 935/1863] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:24 2022] PUT /identity/v3/projects/e789f0ff432a4b65b56ae384889c2fa0/users/7550ea5a5d264d029ede1ff5f3e13f97/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:24.202855 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99907fc5-1c78-4f71-966e-ecbef9b5f182 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:24.213825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99907fc5-1c78-4f71-966e-ecbef9b5f182 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:24.285157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6fdb4f-c9a7-4820-81d2-9b1c716fca8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:24.285907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fdb4f-c9a7-4820-81d2-9b1c716fca8c admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:24.292297 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fdb4f-c9a7-4820-81d2-9b1c716fca8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:24.292297 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fdb4f-c9a7-4820-81d2-9b1c716fca8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:24.292297 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6fdb4f-c9a7-4820-81d2-9b1c716fca8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:24.292297 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6fdb4f-c9a7-4820-81d2-9b1c716fca8c admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:24.302573 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 929/1864] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:24.312438 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a00b4d-6571-47f8-9bd3-ffdc297fba57 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:24.325902 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a00b4d-6571-47f8-9bd3-ffdc297fba57 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:24.341309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059cf709-619b-4c01-a391-bf6272dbd04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:24.342572 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059cf709-619b-4c01-a391-bf6272dbd04b admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:24.343920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059cf709-619b-4c01-a391-bf6272dbd04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:24.343920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059cf709-619b-4c01-a391-bf6272dbd04b admin admin] PATH_INFO: `/v3/projects/e789f0ff432a4b65b56ae384889c2fa0/users/7550ea5a5d264d029ede1ff5f3e13f97/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:24.346717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059cf709-619b-4c01-a391-bf6272dbd04b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e789f0ff432a4b65b56ae384889c2fa0, user_id=7550ea5a5d264d029ede1ff5f3e13f97, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:24.350529 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059cf709-619b-4c01-a391-bf6272dbd04b admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:24.367043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 936/1865] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:24 2022] PUT /identity/v3/projects/e789f0ff432a4b65b56ae384889c2fa0/users/7550ea5a5d264d029ede1ff5f3e13f97/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:24.378277 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82b977-1a45-4e85-a99d-04bd2c81e93d None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:24.378667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82b977-1a45-4e85-a99d-04bd2c81e93d None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:24.378820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82b977-1a45-4e85-a99d-04bd2c81e93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:24.433692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-2b82b977-1a45-4e85-a99d-04bd2c81e93d None None] MFA Rules not processed for user `7550ea5a5d264d029ede1ff5f3e13f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:24.434963 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-2b82b977-1a45-4e85-a99d-04bd2c81e93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:24.542618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 930/1866] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:24.657569 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059cf709-619b-4c01-a391-bf6272dbd04b admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:24.731128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059cf709-619b-4c01-a391-bf6272dbd04b admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:24.735082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed342c34-05fd-492a-9a80-9ba6fa233020 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:24.735815 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed342c34-05fd-492a-9a80-9ba6fa233020 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:24.736191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed342c34-05fd-492a-9a80-9ba6fa233020 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:24.736543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed342c34-05fd-492a-9a80-9ba6fa233020 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:24.749237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed342c34-05fd-492a-9a80-9ba6fa233020 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:24.751391 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed342c34-05fd-492a-9a80-9ba6fa233020 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:24.764698 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 937/1867] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:17:25.005820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82b977-1a45-4e85-a99d-04bd2c81e93d None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:25.015822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82b977-1a45-4e85-a99d-04bd2c81e93d None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:25.019490 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203878d-39f2-44b8-a75b-8f30411c0dcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:25.022656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203878d-39f2-44b8-a75b-8f30411c0dcf service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:25.022656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203878d-39f2-44b8-a75b-8f30411c0dcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:25.022656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203878d-39f2-44b8-a75b-8f30411c0dcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:25.090148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5203878d-39f2-44b8-a75b-8f30411c0dcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:25.091657 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5203878d-39f2-44b8-a75b-8f30411c0dcf service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:25.102216 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 931/1868] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:17:31.559414 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed342c34-05fd-492a-9a80-9ba6fa233020 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:31.562147 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-ed342c34-05fd-492a-9a80-9ba6fa233020 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:31.572187 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed342c34-05fd-492a-9a80-9ba6fa233020 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:31.576651 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ff31e-f0dc-4193-8dfe-f56342b763f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:31.577287 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ff31e-f0dc-4193-8dfe-f56342b763f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:31.577567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ff31e-f0dc-4193-8dfe-f56342b763f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:31.577896 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ff31e-f0dc-4193-8dfe-f56342b763f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:31.642326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ff31e-f0dc-4193-8dfe-f56342b763f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:31.643850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ff31e-f0dc-4193-8dfe-f56342b763f7 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:31.655940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 938/1869] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:17:37.111324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203878d-39f2-44b8-a75b-8f30411c0dcf service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:37.200626 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203878d-39f2-44b8-a75b-8f30411c0dcf service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:37.205304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088a748-a437-4c70-90c7-42fa743762cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:37.206243 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088a748-a437-4c70-90c7-42fa743762cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:37.206531 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088a748-a437-4c70-90c7-42fa743762cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:37.207270 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088a748-a437-4c70-90c7-42fa743762cd admin admin] PATH_INFO: `/v3/users/156ae7d75dd64d82b30214f345f63513` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:37.208098 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088a748-a437-4c70-90c7-42fa743762cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=156ae7d75dd64d82b30214f345f63513)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:37.210069 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088a748-a437-4c70-90c7-42fa743762cd admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:37.399497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-7088a748-a437-4c70-90c7-42fa743762cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '156ae7d75dd64d82b30214f345f63513'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:37.400084 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-7088a748-a437-4c70-90c7-42fa743762cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '156ae7d75dd64d82b30214f345f63513'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:37.414323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-7088a748-a437-4c70-90c7-42fa743762cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '156ae7d75dd64d82b30214f345f63513'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:37.414967 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-7088a748-a437-4c70-90c7-42fa743762cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '156ae7d75dd64d82b30214f345f63513'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:37.422379 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-7088a748-a437-4c70-90c7-42fa743762cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '156ae7d75dd64d82b30214f345f63513'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:37.530546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 932/1870] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:37 2022] DELETE /identity/v3/users/156ae7d75dd64d82b30214f345f63513 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:37.535273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825171e-8eed-4eed-bbd1-2cdd02dd0eda None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:37.535791 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825171e-8eed-4eed-bbd1-2cdd02dd0eda None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:37.535977 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825171e-8eed-4eed-bbd1-2cdd02dd0eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:37.553511 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-2825171e-8eed-4eed-bbd1-2cdd02dd0eda None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:37.555019 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-2825171e-8eed-4eed-bbd1-2cdd02dd0eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:37.619083 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 939/1871] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:37.625634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088a748-a437-4c70-90c7-42fa743762cd admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:37.627075 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-7088a748-a437-4c70-90c7-42fa743762cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:37.699730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088a748-a437-4c70-90c7-42fa743762cd admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:37.705697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82374507-5275-4a55-ba21-050772d6af92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:37.706411 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82374507-5275-4a55-ba21-050772d6af92 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:37.706697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82374507-5275-4a55-ba21-050772d6af92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:37.707000 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82374507-5275-4a55-ba21-050772d6af92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:37.725738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82374507-5275-4a55-ba21-050772d6af92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:37.727655 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82374507-5275-4a55-ba21-050772d6af92 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:37.741288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 933/1872] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:17:38.083069 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825171e-8eed-4eed-bbd1-2cdd02dd0eda None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:38.084569 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-2825171e-8eed-4eed-bbd1-2cdd02dd0eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:38.100182 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825171e-8eed-4eed-bbd1-2cdd02dd0eda None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:38.107429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104e165-deaf-4497-a560-e366b4dc2505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:38.107429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104e165-deaf-4497-a560-e366b4dc2505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:38.107429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104e165-deaf-4497-a560-e366b4dc2505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:38.108036 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104e165-deaf-4497-a560-e366b4dc2505 admin admin] PATH_INFO: `/v3/users/fd594cb249d6432a8ff3a768e74f982e` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:38.109834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104e165-deaf-4497-a560-e366b4dc2505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd594cb249d6432a8ff3a768e74f982e)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:38.113328 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104e165-deaf-4497-a560-e366b4dc2505 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:38.292210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2104e165-deaf-4497-a560-e366b4dc2505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd594cb249d6432a8ff3a768e74f982e'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:38.293046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2104e165-deaf-4497-a560-e366b4dc2505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd594cb249d6432a8ff3a768e74f982e'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:38.294008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2104e165-deaf-4497-a560-e366b4dc2505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd594cb249d6432a8ff3a768e74f982e'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:38.316756 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2104e165-deaf-4497-a560-e366b4dc2505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd594cb249d6432a8ff3a768e74f982e'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:38.324197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2104e165-deaf-4497-a560-e366b4dc2505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd594cb249d6432a8ff3a768e74f982e'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:38.364035 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82374507-5275-4a55-ba21-050772d6af92 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:38.366253 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-82374507-5275-4a55-ba21-050772d6af92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:38.412888 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 940/1873] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:38 2022] DELETE /identity/v3/users/fd594cb249d6432a8ff3a768e74f982e => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:38.414279 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82374507-5275-4a55-ba21-050772d6af92 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:38.415196 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffcbba-1fd0-488d-8d0c-3cbb36a60b10 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:38.415601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffcbba-1fd0-488d-8d0c-3cbb36a60b10 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:38.419014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffcbba-1fd0-488d-8d0c-3cbb36a60b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:38.455304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-c7ffcbba-1fd0-488d-8d0c-3cbb36a60b10 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:38.456650 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-c7ffcbba-1fd0-488d-8d0c-3cbb36a60b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:38.493782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab729f-2107-477e-8a70-9f2f22a811e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:38.495116 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab729f-2107-477e-8a70-9f2f22a811e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:38.496022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab729f-2107-477e-8a70-9f2f22a811e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:38.496400 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab729f-2107-477e-8a70-9f2f22a811e0 admin admin] PATH_INFO: `/v3/projects/b367870b55cf42b19eec3b14a9bfc8aa` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:38.497969 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab729f-2107-477e-8a70-9f2f22a811e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b367870b55cf42b19eec3b14a9bfc8aa)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:38.501288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab729f-2107-477e-8a70-9f2f22a811e0 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:38.517999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 941/1874] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:38.538420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ffcbba-1fd0-488d-8d0c-3cbb36a60b10 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:38.540164 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-c7ffcbba-1fd0-488d-8d0c-3cbb36a60b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:38.597124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ffcbba-1fd0-488d-8d0c-3cbb36a60b10 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:38.600919 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e830c-7f3d-49f1-8eed-3be26fd888a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:38.601748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e830c-7f3d-49f1-8eed-3be26fd888a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:38.602066 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e830c-7f3d-49f1-8eed-3be26fd888a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:38.603053 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e830c-7f3d-49f1-8eed-3be26fd888a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:38.612998 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e830c-7f3d-49f1-8eed-3be26fd888a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:38.614610 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e830c-7f3d-49f1-8eed-3be26fd888a3 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:38.625886 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 942/1875] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:17:38.689491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-15ab729f-2107-477e-8a70-9f2f22a811e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b367870b55cf42b19eec3b14a9bfc8aa', 'request_id': 'req-15ab729f-2107-477e-8a70-9f2f22a811e0'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:38.775859 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-15ab729f-2107-477e-8a70-9f2f22a811e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b367870b55cf42b19eec3b14a9bfc8aa', 'request_id': 'req-15ab729f-2107-477e-8a70-9f2f22a811e0'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:38.782939 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-15ab729f-2107-477e-8a70-9f2f22a811e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b367870b55cf42b19eec3b14a9bfc8aa', 'request_id': 'req-15ab729f-2107-477e-8a70-9f2f22a811e0'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:38.803738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-15ab729f-2107-477e-8a70-9f2f22a811e0 admin admin] The token cache is being invalidate because project b367870b55cf42b19eec3b14a9bfc8aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:38.804708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-15ab729f-2107-477e-8a70-9f2f22a811e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:38.805520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-15ab729f-2107-477e-8a70-9f2f22a811e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:38.807844 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 934/1876] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:38 2022] DELETE /identity/v3/projects/b367870b55cf42b19eec3b14a9bfc8aa => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:38.812622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e830c-7f3d-49f1-8eed-3be26fd888a3 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:38.814298 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-c83e830c-7f3d-49f1-8eed-3be26fd888a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:38.831120 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e830c-7f3d-49f1-8eed-3be26fd888a3 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:38.911933 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87843bf6-e2a3-40ab-83de-673713b262e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:38.917675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87843bf6-e2a3-40ab-83de-673713b262e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:38.917675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87843bf6-e2a3-40ab-83de-673713b262e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:38.917675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87843bf6-e2a3-40ab-83de-673713b262e8 admin admin] PATH_INFO: `/v3/users/7550ea5a5d264d029ede1ff5f3e13f97` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:38.917675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87843bf6-e2a3-40ab-83de-673713b262e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7550ea5a5d264d029ede1ff5f3e13f97)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:38.921054 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87843bf6-e2a3-40ab-83de-673713b262e8 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:39.067697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab729f-2107-477e-8a70-9f2f22a811e0 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:39.069155 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-15ab729f-2107-477e-8a70-9f2f22a811e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:39.082457 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab729f-2107-477e-8a70-9f2f22a811e0 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:39.087719 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91d963-6d82-41b9-bacc-29efc09d8b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:39.088583 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91d963-6d82-41b9-bacc-29efc09d8b52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:39.088859 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91d963-6d82-41b9-bacc-29efc09d8b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:39.089124 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91d963-6d82-41b9-bacc-29efc09d8b52 admin admin] PATH_INFO: `/v3/projects/62ae41513d9d4505a06c4710f82d48ba` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:39.090351 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91d963-6d82-41b9-bacc-29efc09d8b52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62ae41513d9d4505a06c4710f82d48ba)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:39.092242 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91d963-6d82-41b9-bacc-29efc09d8b52 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:39.143201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-87843bf6-e2a3-40ab-83de-673713b262e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7550ea5a5d264d029ede1ff5f3e13f97'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:39.143912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-87843bf6-e2a3-40ab-83de-673713b262e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7550ea5a5d264d029ede1ff5f3e13f97'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:39.144597 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-87843bf6-e2a3-40ab-83de-673713b262e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7550ea5a5d264d029ede1ff5f3e13f97'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:39.156829 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-87843bf6-e2a3-40ab-83de-673713b262e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7550ea5a5d264d029ede1ff5f3e13f97'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:39.164537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-87843bf6-e2a3-40ab-83de-673713b262e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7550ea5a5d264d029ede1ff5f3e13f97'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:39.200217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1b91d963-6d82-41b9-bacc-29efc09d8b52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62ae41513d9d4505a06c4710f82d48ba', 'request_id': 'req-1b91d963-6d82-41b9-bacc-29efc09d8b52'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:39.259835 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 943/1877] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:38 2022] DELETE /identity/v3/users/7550ea5a5d264d029ede1ff5f3e13f97 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:39.305819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1b91d963-6d82-41b9-bacc-29efc09d8b52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62ae41513d9d4505a06c4710f82d48ba', 'request_id': 'req-1b91d963-6d82-41b9-bacc-29efc09d8b52'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:39.323814 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1b91d963-6d82-41b9-bacc-29efc09d8b52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62ae41513d9d4505a06c4710f82d48ba', 'request_id': 'req-1b91d963-6d82-41b9-bacc-29efc09d8b52'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:39.341737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1b91d963-6d82-41b9-bacc-29efc09d8b52 admin admin] The token cache is being invalidate because project 62ae41513d9d4505a06c4710f82d48ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:39.342270 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1b91d963-6d82-41b9-bacc-29efc09d8b52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:39.343010 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1b91d963-6d82-41b9-bacc-29efc09d8b52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:39.346119 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 935/1878] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:39 2022] DELETE /identity/v3/projects/62ae41513d9d4505a06c4710f82d48ba => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:39.350948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87843bf6-e2a3-40ab-83de-673713b262e8 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:39.353099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-87843bf6-e2a3-40ab-83de-673713b262e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:39.452815 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87843bf6-e2a3-40ab-83de-673713b262e8 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:39.458017 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b72baa-041f-428a-905e-0336de5e6f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:39.458931 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b72baa-041f-428a-905e-0336de5e6f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:39.459285 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b72baa-041f-428a-905e-0336de5e6f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:39.459632 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b72baa-041f-428a-905e-0336de5e6f9a admin admin] PATH_INFO: `/v3/users/62487501d4364e5284c8ed829f9223f0` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:39.461035 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b72baa-041f-428a-905e-0336de5e6f9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=62487501d4364e5284c8ed829f9223f0)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:39.463098 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b72baa-041f-428a-905e-0336de5e6f9a admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:39.612026 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b8b72baa-041f-428a-905e-0336de5e6f9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62487501d4364e5284c8ed829f9223f0'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:39.615310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b8b72baa-041f-428a-905e-0336de5e6f9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62487501d4364e5284c8ed829f9223f0'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:39.615310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b8b72baa-041f-428a-905e-0336de5e6f9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62487501d4364e5284c8ed829f9223f0'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:39.625356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b8b72baa-041f-428a-905e-0336de5e6f9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62487501d4364e5284c8ed829f9223f0'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:39.632771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b8b72baa-041f-428a-905e-0336de5e6f9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62487501d4364e5284c8ed829f9223f0'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:39.712613 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 944/1879] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:39 2022] DELETE /identity/v3/users/62487501d4364e5284c8ed829f9223f0 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:39.832272 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91d963-6d82-41b9-bacc-29efc09d8b52 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:39.834318 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1b91d963-6d82-41b9-bacc-29efc09d8b52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:39.934571 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91d963-6d82-41b9-bacc-29efc09d8b52 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:39.993390 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d0fb0-dc54-4ca6-aebc-855e33beb1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:39.994283 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d0fb0-dc54-4ca6-aebc-855e33beb1c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:39.994593 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d0fb0-dc54-4ca6-aebc-855e33beb1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:39.994868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d0fb0-dc54-4ca6-aebc-855e33beb1c5 admin admin] PATH_INFO: `/v3/projects/e789f0ff432a4b65b56ae384889c2fa0` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:39.996046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d0fb0-dc54-4ca6-aebc-855e33beb1c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e789f0ff432a4b65b56ae384889c2fa0)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:39.997924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d0fb0-dc54-4ca6-aebc-855e33beb1c5 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:40.121623 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-737d0fb0-dc54-4ca6-aebc-855e33beb1c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e789f0ff432a4b65b56ae384889c2fa0', 'request_id': 'req-737d0fb0-dc54-4ca6-aebc-855e33beb1c5'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:40.173796 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-737d0fb0-dc54-4ca6-aebc-855e33beb1c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e789f0ff432a4b65b56ae384889c2fa0', 'request_id': 'req-737d0fb0-dc54-4ca6-aebc-855e33beb1c5'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:40.178560 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-737d0fb0-dc54-4ca6-aebc-855e33beb1c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e789f0ff432a4b65b56ae384889c2fa0', 'request_id': 'req-737d0fb0-dc54-4ca6-aebc-855e33beb1c5'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:40.190862 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-737d0fb0-dc54-4ca6-aebc-855e33beb1c5 admin admin] The token cache is being invalidate because project e789f0ff432a4b65b56ae384889c2fa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:40.191148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-737d0fb0-dc54-4ca6-aebc-855e33beb1c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:40.191813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-737d0fb0-dc54-4ca6-aebc-855e33beb1c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:40.193447 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 936/1880] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:39 2022] DELETE /identity/v3/projects/e789f0ff432a4b65b56ae384889c2fa0 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:40.198326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caded3a9-036d-459e-8c46-73dbe2999b45 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:40.198625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caded3a9-036d-459e-8c46-73dbe2999b45 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:40.199898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caded3a9-036d-459e-8c46-73dbe2999b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:40.221256 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d0fb0-dc54-4ca6-aebc-855e33beb1c5 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:40.222534 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-caded3a9-036d-459e-8c46-73dbe2999b45 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:40.223087 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-737d0fb0-dc54-4ca6-aebc-855e33beb1c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:40.223742 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-caded3a9-036d-459e-8c46-73dbe2999b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:40.287244 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 945/1881] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:40.294830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a2587-ff00-4c6d-b79f-01fcb3152bc5 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:40.295124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a2587-ff00-4c6d-b79f-01fcb3152bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:40.295464 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a2587-ff00-4c6d-b79f-01fcb3152bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:40.319128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d0fb0-dc54-4ca6-aebc-855e33beb1c5 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:40.319338 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-e09a2587-ff00-4c6d-b79f-01fcb3152bc5 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:40.319338 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-e09a2587-ff00-4c6d-b79f-01fcb3152bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:40.319518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa711d9e-fbff-4e7a-b7c3-d1883d424829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:40.319761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa711d9e-fbff-4e7a-b7c3-d1883d424829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:40.320053 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa711d9e-fbff-4e7a-b7c3-d1883d424829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:40.320322 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa711d9e-fbff-4e7a-b7c3-d1883d424829 admin admin] PATH_INFO: `/v3/projects/52768df123974dccb42a0b088dabd99a` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:40.321648 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa711d9e-fbff-4e7a-b7c3-d1883d424829 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52768df123974dccb42a0b088dabd99a)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:40.323819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa711d9e-fbff-4e7a-b7c3-d1883d424829 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:40.325875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 946/1882] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:40.330660 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a2587-ff00-4c6d-b79f-01fcb3152bc5 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:40.339986 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a2587-ff00-4c6d-b79f-01fcb3152bc5 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:40.348008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40f910-26e1-4df7-8d8f-32e722f9d455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:40.349198 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40f910-26e1-4df7-8d8f-32e722f9d455 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:40.349764 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40f910-26e1-4df7-8d8f-32e722f9d455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:40.350494 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40f910-26e1-4df7-8d8f-32e722f9d455 admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:40.351164 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40f910-26e1-4df7-8d8f-32e722f9d455 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:17:40.351896 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40f910-26e1-4df7-8d8f-32e722f9d455 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:40.353746 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40f910-26e1-4df7-8d8f-32e722f9d455 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:40.366896 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 947/1883] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 20:17:40.371547 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40f910-26e1-4df7-8d8f-32e722f9d455 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:40.381170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40f910-26e1-4df7-8d8f-32e722f9d455 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:40.386109 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee864edf-0fb1-4ef3-a518-8d63d7649a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:40.386794 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee864edf-0fb1-4ef3-a518-8d63d7649a69 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:40.387083 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee864edf-0fb1-4ef3-a518-8d63d7649a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:40.387364 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee864edf-0fb1-4ef3-a518-8d63d7649a69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:40.388063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee864edf-0fb1-4ef3-a518-8d63d7649a69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:40.390043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee864edf-0fb1-4ef3-a518-8d63d7649a69 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:40.423735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-fa711d9e-fbff-4e7a-b7c3-d1883d424829 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52768df123974dccb42a0b088dabd99a', 'request_id': 'req-fa711d9e-fbff-4e7a-b7c3-d1883d424829'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:40.517936 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 948/1884] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:40 2022] POST /identity/v3/projects => generated 367 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:17:40.518752 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-fa711d9e-fbff-4e7a-b7c3-d1883d424829 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52768df123974dccb42a0b088dabd99a', 'request_id': 'req-fa711d9e-fbff-4e7a-b7c3-d1883d424829'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:40.518923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee864edf-0fb1-4ef3-a518-8d63d7649a69 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:40.526122 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-fa711d9e-fbff-4e7a-b7c3-d1883d424829 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52768df123974dccb42a0b088dabd99a', 'request_id': 'req-fa711d9e-fbff-4e7a-b7c3-d1883d424829'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:40.538040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-fa711d9e-fbff-4e7a-b7c3-d1883d424829 admin admin] The token cache is being invalidate because project 52768df123974dccb42a0b088dabd99a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:40.538479 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-fa711d9e-fbff-4e7a-b7c3-d1883d424829 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:40.539193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-fa711d9e-fbff-4e7a-b7c3-d1883d424829 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:40.541085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 937/1885] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:40 2022] DELETE /identity/v3/projects/52768df123974dccb42a0b088dabd99a => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:40.551986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92f628-a946-4c71-bc53-4e0d8807ae4d None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:40.552352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92f628-a946-4c71-bc53-4e0d8807ae4d None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:40.552698 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92f628-a946-4c71-bc53-4e0d8807ae4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:40.573632 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-9b92f628-a946-4c71-bc53-4e0d8807ae4d None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:40.574828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-9b92f628-a946-4c71-bc53-4e0d8807ae4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:40.582715 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee864edf-0fb1-4ef3-a518-8d63d7649a69 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:40.585891 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 938/1886] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:40.587694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eabb2f8-9cb5-471d-9247-c29923140f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:40.588451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabb2f8-9cb5-471d-9247-c29923140f8c admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:40.588733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabb2f8-9cb5-471d-9247-c29923140f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:40.589008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabb2f8-9cb5-471d-9247-c29923140f8c admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:40.589751 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eabb2f8-9cb5-471d-9247-c29923140f8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:40.591704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eabb2f8-9cb5-471d-9247-c29923140f8c admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:40.599266 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98379f2d-6be6-41cc-862e-e5c95a7580f7 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:40.599562 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98379f2d-6be6-41cc-862e-e5c95a7580f7 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:40.602122 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98379f2d-6be6-41cc-862e-e5c95a7580f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:40.621978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-98379f2d-6be6-41cc-862e-e5c95a7580f7 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:40.623533 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-98379f2d-6be6-41cc-862e-e5c95a7580f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:40.635306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 939/1887] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:40.636314 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98379f2d-6be6-41cc-862e-e5c95a7580f7 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:40.648974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98379f2d-6be6-41cc-862e-e5c95a7580f7 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:40.654622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434231c-545f-4b2f-b8a2-34d685bd2003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:40.656077 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434231c-545f-4b2f-b8a2-34d685bd2003 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:40.656179 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434231c-545f-4b2f-b8a2-34d685bd2003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:40.656390 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434231c-545f-4b2f-b8a2-34d685bd2003 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:40.657149 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434231c-545f-4b2f-b8a2-34d685bd2003 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:17:40.657429 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434231c-545f-4b2f-b8a2-34d685bd2003 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:40.659155 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434231c-545f-4b2f-b8a2-34d685bd2003 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:40.671507 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 940/1888] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:17:40.675806 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434231c-545f-4b2f-b8a2-34d685bd2003 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:40.688359 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434231c-545f-4b2f-b8a2-34d685bd2003 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:40.694118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 949/1889] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:40 2022] POST /identity/v3/users => generated 284 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:17:40.697799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56e49f-8e42-4c8a-ad11-33d365c710da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:40.697799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56e49f-8e42-4c8a-ad11-33d365c710da admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:40.698008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eabb2f8-9cb5-471d-9247-c29923140f8c admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:40.698134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56e49f-8e42-4c8a-ad11-33d365c710da admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:40.698134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56e49f-8e42-4c8a-ad11-33d365c710da admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:40.698134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56e49f-8e42-4c8a-ad11-33d365c710da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:40.698421 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-1eabb2f8-9cb5-471d-9247-c29923140f8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:40.699097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56e49f-8e42-4c8a-ad11-33d365c710da admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:40.711440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eabb2f8-9cb5-471d-9247-c29923140f8c admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:40.717938 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e801f-b491-48ab-961b-2c34491c741d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:40.718656 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e801f-b491-48ab-961b-2c34491c741d admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:40.718943 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e801f-b491-48ab-961b-2c34491c741d admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:40.723943 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e801f-b491-48ab-961b-2c34491c741d admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:40.723943 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e801f-b491-48ab-961b-2c34491c741d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:40.723943 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e801f-b491-48ab-961b-2c34491c741d admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:40.734307 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 950/1890] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:40.739179 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e801f-b491-48ab-961b-2c34491c741d admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:40.751800 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e801f-b491-48ab-961b-2c34491c741d admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:40.756809 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76719993-2846-45fe-b886-b66c59a9851b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:40.757498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76719993-2846-45fe-b886-b66c59a9851b admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:40.757785 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76719993-2846-45fe-b886-b66c59a9851b admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:40.758223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76719993-2846-45fe-b886-b66c59a9851b admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:40.759035 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76719993-2846-45fe-b886-b66c59a9851b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:40.760814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76719993-2846-45fe-b886-b66c59a9851b admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:40.772908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 951/1891] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:40.777068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76719993-2846-45fe-b886-b66c59a9851b admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:40.787770 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76719993-2846-45fe-b886-b66c59a9851b admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:40.793219 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c5253-e800-43ab-9520-eb86b79d666a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:40.794233 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c5253-e800-43ab-9520-eb86b79d666a admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:40.794521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c5253-e800-43ab-9520-eb86b79d666a admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:40.794797 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c5253-e800-43ab-9520-eb86b79d666a admin admin] PATH_INFO: `/v3/projects/b4bdbfc1c51d407b9f45654f9f9169d7/users/875e656570b34117a8780c37e291964f/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:40.816719 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c5253-e800-43ab-9520-eb86b79d666a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4bdbfc1c51d407b9f45654f9f9169d7, user_id=875e656570b34117a8780c37e291964f, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:40.818482 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 941/1892] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:40 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:17:40.819375 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56e49f-8e42-4c8a-ad11-33d365c710da admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:40.827352 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c5253-e800-43ab-9520-eb86b79d666a admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:40.888703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56e49f-8e42-4c8a-ad11-33d365c710da admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:40.893935 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c020e-9c1e-4b0c-ab62-c590b0a30478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:40.894644 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c020e-9c1e-4b0c-ab62-c590b0a30478 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:40.901772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c020e-9c1e-4b0c-ab62-c590b0a30478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:40.901772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c020e-9c1e-4b0c-ab62-c590b0a30478 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:40.901772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c020e-9c1e-4b0c-ab62-c590b0a30478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:40.901772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c020e-9c1e-4b0c-ab62-c590b0a30478 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:40.930898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 952/1893] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:40 2022] PUT /identity/v3/projects/b4bdbfc1c51d407b9f45654f9f9169d7/users/875e656570b34117a8780c37e291964f/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:40.942962 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6361c1-d7fe-4f92-a933-f65d560c0c7f None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:40.943254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6361c1-d7fe-4f92-a933-f65d560c0c7f None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:40.943533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6361c1-d7fe-4f92-a933-f65d560c0c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:40.979306 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-ee6361c1-d7fe-4f92-a933-f65d560c0c7f None None] MFA Rules not processed for user `875e656570b34117a8780c37e291964f`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:40.980852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-ee6361c1-d7fe-4f92-a933-f65d560c0c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:41.028414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 942/1894] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:40 2022] POST /identity/v3/users => generated 283 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:41.032532 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c020e-9c1e-4b0c-ab62-c590b0a30478 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:41.043822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c020e-9c1e-4b0c-ab62-c590b0a30478 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:41.059871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 953/1895] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:40 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:41.072092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6361c1-d7fe-4f92-a933-f65d560c0c7f None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:41.073496 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-ee6361c1-d7fe-4f92-a933-f65d560c0c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:41.115076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269cd5c-13c0-4568-baa2-48276032d1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:41.116328 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269cd5c-13c0-4568-baa2-48276032d1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:41.116721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269cd5c-13c0-4568-baa2-48276032d1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:41.117048 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269cd5c-13c0-4568-baa2-48276032d1c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:41.117864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269cd5c-13c0-4568-baa2-48276032d1c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:41.119648 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269cd5c-13c0-4568-baa2-48276032d1c5 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:41.130338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 943/1896] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:17:41.134209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269cd5c-13c0-4568-baa2-48276032d1c5 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:41.136495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6361c1-d7fe-4f92-a933-f65d560c0c7f None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:41.140694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47305999-91de-4605-9642-09979e654b2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:41.141441 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47305999-91de-4605-9642-09979e654b2c service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:41.141725 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47305999-91de-4605-9642-09979e654b2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:41.142000 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47305999-91de-4605-9642-09979e654b2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:41.144171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269cd5c-13c0-4568-baa2-48276032d1c5 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:41.148784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3b875-ad5b-4bdb-b3f4-4992c268a763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:41.149422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3b875-ad5b-4bdb-b3f4-4992c268a763 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:41.149708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3b875-ad5b-4bdb-b3f4-4992c268a763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:41.149985 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3b875-ad5b-4bdb-b3f4-4992c268a763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:41.150743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3b875-ad5b-4bdb-b3f4-4992c268a763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:41.150885 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47305999-91de-4605-9642-09979e654b2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:41.152322 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3b875-ad5b-4bdb-b3f4-4992c268a763 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:41.152435 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47305999-91de-4605-9642-09979e654b2c service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:41.161322 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 954/1897] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:41 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:17:41.161979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 944/1898] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:17:41.168282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47305999-91de-4605-9642-09979e654b2c service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:41.180784 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47305999-91de-4605-9642-09979e654b2c service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:41.181471 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3b875-ad5b-4bdb-b3f4-4992c268a763 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:41.185748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c6873-0cef-4c17-b315-0ef406de9f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:41.187422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c6873-0cef-4c17-b315-0ef406de9f53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:41.187808 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c6873-0cef-4c17-b315-0ef406de9f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:41.193402 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c6873-0cef-4c17-b315-0ef406de9f53 admin admin] PATH_INFO: `/v3/projects/48c1f68b49d44c62a39426915ec1b659/users/f937d5e4b52646b49cea6b2d5b267069/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:41.194237 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3b875-ad5b-4bdb-b3f4-4992c268a763 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:41.200728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0147df-040e-4991-ad39-e27f428245fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:41.201629 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0147df-040e-4991-ad39-e27f428245fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:41.202517 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0147df-040e-4991-ad39-e27f428245fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:41.202803 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0147df-040e-4991-ad39-e27f428245fb admin admin] PATH_INFO: `/v3/users/875e656570b34117a8780c37e291964f` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:41.204157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0147df-040e-4991-ad39-e27f428245fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=875e656570b34117a8780c37e291964f)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:41.206249 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0147df-040e-4991-ad39-e27f428245fb admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:41.206632 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c6873-0cef-4c17-b315-0ef406de9f53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48c1f68b49d44c62a39426915ec1b659, user_id=f937d5e4b52646b49cea6b2d5b267069, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:41.209414 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c6873-0cef-4c17-b315-0ef406de9f53 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:41.296935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 955/1899] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:41 2022] PUT /identity/v3/projects/48c1f68b49d44c62a39426915ec1b659/users/f937d5e4b52646b49cea6b2d5b267069/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:41.307945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa262f0a-6f29-4273-aee1-d63168826634 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:41.308240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa262f0a-6f29-4273-aee1-d63168826634 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:41.308857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa262f0a-6f29-4273-aee1-d63168826634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:41.335571 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-aa262f0a-6f29-4273-aee1-d63168826634 None None] MFA Rules not processed for user `f937d5e4b52646b49cea6b2d5b267069`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:41.336758 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-aa262f0a-6f29-4273-aee1-d63168826634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:41.385952 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1e0147df-040e-4991-ad39-e27f428245fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '875e656570b34117a8780c37e291964f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:41.386693 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1e0147df-040e-4991-ad39-e27f428245fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '875e656570b34117a8780c37e291964f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:41.398575 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1e0147df-040e-4991-ad39-e27f428245fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '875e656570b34117a8780c37e291964f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:41.399370 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1e0147df-040e-4991-ad39-e27f428245fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '875e656570b34117a8780c37e291964f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:41.400172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 956/1900] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:41.404591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa262f0a-6f29-4273-aee1-d63168826634 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:41.409855 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1e0147df-040e-4991-ad39-e27f428245fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '875e656570b34117a8780c37e291964f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:41.467751 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa262f0a-6f29-4273-aee1-d63168826634 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:41.472978 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c34540-64c6-4eea-9b56-f01994f8f9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:41.473648 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c34540-64c6-4eea-9b56-f01994f8f9f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:41.473962 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c34540-64c6-4eea-9b56-f01994f8f9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:41.474276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c34540-64c6-4eea-9b56-f01994f8f9f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:41.475038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c34540-64c6-4eea-9b56-f01994f8f9f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:41.477195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c34540-64c6-4eea-9b56-f01994f8f9f7 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:41.483798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 945/1901] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:41 2022] DELETE /identity/v3/users/875e656570b34117a8780c37e291964f => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:41.488221 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613f740-404e-44de-b003-0cc6cb5d38cd None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:41.488475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613f740-404e-44de-b003-0cc6cb5d38cd None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:41.488824 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613f740-404e-44de-b003-0cc6cb5d38cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:41.506867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-6613f740-404e-44de-b003-0cc6cb5d38cd None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:41.508088 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-6613f740-404e-44de-b003-0cc6cb5d38cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:41.580277 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 946/1902] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:41.590034 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613f740-404e-44de-b003-0cc6cb5d38cd None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:41.591383 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-6613f740-404e-44de-b003-0cc6cb5d38cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:41.600361 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 957/1903] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:41 2022] POST /identity/v3/projects => generated 373 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:17:41.604194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c34540-64c6-4eea-9b56-f01994f8f9f7 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:41.605877 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b3c34540-64c6-4eea-9b56-f01994f8f9f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:41.649600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613f740-404e-44de-b003-0cc6cb5d38cd None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:41.671600 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c34540-64c6-4eea-9b56-f01994f8f9f7 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:41.676596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530571d-df85-4cef-83b9-a08315007c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:41.677459 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530571d-df85-4cef-83b9-a08315007c68 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:41.677954 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530571d-df85-4cef-83b9-a08315007c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:41.678199 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530571d-df85-4cef-83b9-a08315007c68 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:41.685611 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530571d-df85-4cef-83b9-a08315007c68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:41.685611 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530571d-df85-4cef-83b9-a08315007c68 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:41.700333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6797dd3-399f-4963-a6fa-3a19eea47e56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:41.700979 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6797dd3-399f-4963-a6fa-3a19eea47e56 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:41.701262 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6797dd3-399f-4963-a6fa-3a19eea47e56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:41.701539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6797dd3-399f-4963-a6fa-3a19eea47e56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:41.711093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6797dd3-399f-4963-a6fa-3a19eea47e56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:41.714415 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6797dd3-399f-4963-a6fa-3a19eea47e56 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:41.721827 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 947/1904] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:17:41.808406 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 958/1905] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:41 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:17:41.810308 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6797dd3-399f-4963-a6fa-3a19eea47e56 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:41.827775 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6797dd3-399f-4963-a6fa-3a19eea47e56 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:41.828782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0b300-f84c-4d48-bb61-037b7d84c155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:41.829457 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0b300-f84c-4d48-bb61-037b7d84c155 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:41.829742 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0b300-f84c-4d48-bb61-037b7d84c155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:41.830022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0b300-f84c-4d48-bb61-037b7d84c155 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:41.830914 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c0b300-f84c-4d48-bb61-037b7d84c155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:41.832575 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c0b300-f84c-4d48-bb61-037b7d84c155 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:41.842996 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 948/1906] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:41.847238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530571d-df85-4cef-83b9-a08315007c68 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:41.859643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530571d-df85-4cef-83b9-a08315007c68 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:41.864377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49cb84-d344-49fa-8be2-3fd9e180d43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:41.865262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49cb84-d344-49fa-8be2-3fd9e180d43f admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:41.865540 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49cb84-d344-49fa-8be2-3fd9e180d43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:41.865827 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49cb84-d344-49fa-8be2-3fd9e180d43f admin admin] PATH_INFO: `/v3/projects/1d577c7b91544e72a48eea267113c6c2/users/87dc297d6b9943abb45fd4a0d0b6791f/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:41.886834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49cb84-d344-49fa-8be2-3fd9e180d43f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d577c7b91544e72a48eea267113c6c2, user_id=87dc297d6b9943abb45fd4a0d0b6791f, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:41.890898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49cb84-d344-49fa-8be2-3fd9e180d43f admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:41.960327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 959/1907] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:41 2022] PUT /identity/v3/projects/1d577c7b91544e72a48eea267113c6c2/users/87dc297d6b9943abb45fd4a0d0b6791f/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:41.965524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0b300-f84c-4d48-bb61-037b7d84c155 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:41.974702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0b300-f84c-4d48-bb61-037b7d84c155 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:42.049872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b6963-8eea-40e4-a7a2-ac61db1195fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:42.050683 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b6963-8eea-40e4-a7a2-ac61db1195fa admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:42.051026 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b6963-8eea-40e4-a7a2-ac61db1195fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:42.051311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b6963-8eea-40e4-a7a2-ac61db1195fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:42.052050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b6963-8eea-40e4-a7a2-ac61db1195fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:42.053650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b6963-8eea-40e4-a7a2-ac61db1195fa admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:42.065770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 949/1908] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:42.069720 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49cb84-d344-49fa-8be2-3fd9e180d43f admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:42.080173 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49cb84-d344-49fa-8be2-3fd9e180d43f admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:42.084891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37f3e2-5074-433e-a15c-3c898af00865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:42.085762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37f3e2-5074-433e-a15c-3c898af00865 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:42.086046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37f3e2-5074-433e-a15c-3c898af00865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:42.086357 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37f3e2-5074-433e-a15c-3c898af00865 admin admin] PATH_INFO: `/v3/projects/1d577c7b91544e72a48eea267113c6c2/users/87dc297d6b9943abb45fd4a0d0b6791f/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:42.088315 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37f3e2-5074-433e-a15c-3c898af00865 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d577c7b91544e72a48eea267113c6c2, user_id=87dc297d6b9943abb45fd4a0d0b6791f, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:42.091064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37f3e2-5074-433e-a15c-3c898af00865 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:42.102197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 960/1909] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:42 2022] PUT /identity/v3/projects/1d577c7b91544e72a48eea267113c6c2/users/87dc297d6b9943abb45fd4a0d0b6791f/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:42.113292 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a7710-1171-4e5b-945a-f1400180d5a8 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:42.113587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a7710-1171-4e5b-945a-f1400180d5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:42.113863 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a7710-1171-4e5b-945a-f1400180d5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:42.140997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-471a7710-1171-4e5b-945a-f1400180d5a8 None None] MFA Rules not processed for user `87dc297d6b9943abb45fd4a0d0b6791f`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:42.142276 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-471a7710-1171-4e5b-945a-f1400180d5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:42.209256 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 950/1910] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:42.212862 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37f3e2-5074-433e-a15c-3c898af00865 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:42.215612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-0c37f3e2-5074-433e-a15c-3c898af00865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:42.220747 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a7710-1171-4e5b-945a-f1400180d5a8 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:42.222437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-471a7710-1171-4e5b-945a-f1400180d5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:42.281147 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a7710-1171-4e5b-945a-f1400180d5a8 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:42.284857 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1496e2-b1c4-4a44-8583-895687088f82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:42.285502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1496e2-b1c4-4a44-8583-895687088f82 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:42.285780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1496e2-b1c4-4a44-8583-895687088f82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:42.286050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1496e2-b1c4-4a44-8583-895687088f82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:42.288493 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37f3e2-5074-433e-a15c-3c898af00865 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:42.293141 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b4854-b2a0-4180-99c7-883bd8872009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:42.293992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b4854-b2a0-4180-99c7-883bd8872009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:42.294307 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b4854-b2a0-4180-99c7-883bd8872009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:42.294585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b4854-b2a0-4180-99c7-883bd8872009 admin admin] PATH_INFO: `/v3/projects/b4bdbfc1c51d407b9f45654f9f9169d7` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:42.295147 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1496e2-b1c4-4a44-8583-895687088f82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:42.296298 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b4854-b2a0-4180-99c7-883bd8872009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4bdbfc1c51d407b9f45654f9f9169d7)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:42.297093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1496e2-b1c4-4a44-8583-895687088f82 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:42.298209 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b4854-b2a0-4180-99c7-883bd8872009 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:42.306634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 951/1911] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:17:42.479117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6d1b4854-b2a0-4180-99c7-883bd8872009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4bdbfc1c51d407b9f45654f9f9169d7', 'request_id': 'req-6d1b4854-b2a0-4180-99c7-883bd8872009'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:42.568991 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6d1b4854-b2a0-4180-99c7-883bd8872009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4bdbfc1c51d407b9f45654f9f9169d7', 'request_id': 'req-6d1b4854-b2a0-4180-99c7-883bd8872009'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:42.574949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6d1b4854-b2a0-4180-99c7-883bd8872009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4bdbfc1c51d407b9f45654f9f9169d7', 'request_id': 'req-6d1b4854-b2a0-4180-99c7-883bd8872009'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:42.586427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6d1b4854-b2a0-4180-99c7-883bd8872009 admin admin] The token cache is being invalidate because project b4bdbfc1c51d407b9f45654f9f9169d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:42.586715 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6d1b4854-b2a0-4180-99c7-883bd8872009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:42.587371 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6d1b4854-b2a0-4180-99c7-883bd8872009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:42.588935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 961/1912] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:42 2022] DELETE /identity/v3/projects/b4bdbfc1c51d407b9f45654f9f9169d7 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:42.594438 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a5845-a5f3-46dc-909b-313f79ef8e86 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:42.594732 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a5845-a5f3-46dc-909b-313f79ef8e86 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:42.595011 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a5845-a5f3-46dc-909b-313f79ef8e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:42.612859 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-f26a5845-a5f3-46dc-909b-313f79ef8e86 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:42.614151 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f26a5845-a5f3-46dc-909b-313f79ef8e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:42.673859 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 952/1913] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:42.682770 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede4cd2-9292-4fb9-b400-8dc4288a056f None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:42.683108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede4cd2-9292-4fb9-b400-8dc4288a056f None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:42.683549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede4cd2-9292-4fb9-b400-8dc4288a056f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:42.702963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-9ede4cd2-9292-4fb9-b400-8dc4288a056f None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:42.704197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-9ede4cd2-9292-4fb9-b400-8dc4288a056f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:42.712346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 962/1914] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:42.716662 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a5845-a5f3-46dc-909b-313f79ef8e86 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:42.726618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a5845-a5f3-46dc-909b-313f79ef8e86 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:42.731272 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacbe8a-9f9a-4391-80fd-6c72d6944a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:42.731922 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacbe8a-9f9a-4391-80fd-6c72d6944a6b admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:42.732206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacbe8a-9f9a-4391-80fd-6c72d6944a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:42.732483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacbe8a-9f9a-4391-80fd-6c72d6944a6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:42.733244 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacbe8a-9f9a-4391-80fd-6c72d6944a6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:17:42.733552 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacbe8a-9f9a-4391-80fd-6c72d6944a6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:42.735301 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacbe8a-9f9a-4391-80fd-6c72d6944a6b admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:42.746799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 953/1915] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:17:42.751549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede4cd2-9292-4fb9-b400-8dc4288a056f None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:42.760585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede4cd2-9292-4fb9-b400-8dc4288a056f None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:42.766052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0478a3c-def9-48e3-a3a2-a5af58898a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:42.766737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0478a3c-def9-48e3-a3a2-a5af58898a06 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:42.767020 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0478a3c-def9-48e3-a3a2-a5af58898a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:42.767377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0478a3c-def9-48e3-a3a2-a5af58898a06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:42.768090 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0478a3c-def9-48e3-a3a2-a5af58898a06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:42.770010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0478a3c-def9-48e3-a3a2-a5af58898a06 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:42.878810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 963/1916] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:42 2022] POST /identity/v3/projects => generated 371 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:42.880960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacbe8a-9f9a-4391-80fd-6c72d6944a6b admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:42.892629 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacbe8a-9f9a-4391-80fd-6c72d6944a6b admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:42.960717 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db549234-a249-4ed0-940b-d7c778aa73b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:42.961377 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db549234-a249-4ed0-940b-d7c778aa73b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:42.961661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db549234-a249-4ed0-940b-d7c778aa73b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:42.961925 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db549234-a249-4ed0-940b-d7c778aa73b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:42.962703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db549234-a249-4ed0-940b-d7c778aa73b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:42.964594 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db549234-a249-4ed0-940b-d7c778aa73b3 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:43.066070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 954/1917] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:42 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:43.070289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0478a3c-def9-48e3-a3a2-a5af58898a06 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:43.087006 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0478a3c-def9-48e3-a3a2-a5af58898a06 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:43.094462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f2a37-7e75-498e-90be-a2e87ac5e5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:43.095443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f2a37-7e75-498e-90be-a2e87ac5e5ba admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:43.095731 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f2a37-7e75-498e-90be-a2e87ac5e5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:43.096988 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f2a37-7e75-498e-90be-a2e87ac5e5ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:43.098431 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f2a37-7e75-498e-90be-a2e87ac5e5ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:43.100237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f2a37-7e75-498e-90be-a2e87ac5e5ba admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:43.115645 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 964/1918] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:43.119757 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db549234-a249-4ed0-940b-d7c778aa73b3 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:43.131028 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db549234-a249-4ed0-940b-d7c778aa73b3 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:43.136438 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8f7a0-9cbb-410a-b3f5-0df72fc2763a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:43.137221 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8f7a0-9cbb-410a-b3f5-0df72fc2763a admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:43.137570 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8f7a0-9cbb-410a-b3f5-0df72fc2763a admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:43.137911 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8f7a0-9cbb-410a-b3f5-0df72fc2763a admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:43.138804 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8f7a0-9cbb-410a-b3f5-0df72fc2763a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:43.141062 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8f7a0-9cbb-410a-b3f5-0df72fc2763a admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:43.166336 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 955/1919] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:43.172133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f2a37-7e75-498e-90be-a2e87ac5e5ba admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:43.183995 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f2a37-7e75-498e-90be-a2e87ac5e5ba admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:43.195419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d28c2-67d0-4baa-ab51-379361b6a2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:43.206656 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d28c2-67d0-4baa-ab51-379361b6a2a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:43.206962 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d28c2-67d0-4baa-ab51-379361b6a2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:43.208402 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d28c2-67d0-4baa-ab51-379361b6a2a4 admin admin] PATH_INFO: `/v3/projects/a9d63255507b45d285978da72081888a/users/1b5de9557e2341fe9eb05fc3f17bf620/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:43.231561 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d28c2-67d0-4baa-ab51-379361b6a2a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9d63255507b45d285978da72081888a, user_id=1b5de9557e2341fe9eb05fc3f17bf620, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:43.234751 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d28c2-67d0-4baa-ab51-379361b6a2a4 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:43.287323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8f7a0-9cbb-410a-b3f5-0df72fc2763a admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:43.289109 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-9ef8f7a0-9cbb-410a-b3f5-0df72fc2763a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:43.305599 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8f7a0-9cbb-410a-b3f5-0df72fc2763a admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:43.312509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684c457-2410-4062-a03e-4751e767e1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:43.313378 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684c457-2410-4062-a03e-4751e767e1cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:43.313663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684c457-2410-4062-a03e-4751e767e1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:43.313940 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684c457-2410-4062-a03e-4751e767e1cf admin admin] PATH_INFO: `/v3/users/f937d5e4b52646b49cea6b2d5b267069` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:43.314845 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 965/1920] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:43 2022] PUT /identity/v3/projects/a9d63255507b45d285978da72081888a/users/1b5de9557e2341fe9eb05fc3f17bf620/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:43.323726 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684c457-2410-4062-a03e-4751e767e1cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f937d5e4b52646b49cea6b2d5b267069)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:43.323726 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684c457-2410-4062-a03e-4751e767e1cf admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:43.325407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8163f-d6d1-4569-a078-069b500860d9 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:43.325796 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8163f-d6d1-4569-a078-069b500860d9 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:43.326172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8163f-d6d1-4569-a078-069b500860d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:43.352595 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-79d8163f-d6d1-4569-a078-069b500860d9 None None] MFA Rules not processed for user `1b5de9557e2341fe9eb05fc3f17bf620`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:43.353790 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-79d8163f-d6d1-4569-a078-069b500860d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:43.416536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 966/1921] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:43.421120 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8163f-d6d1-4569-a078-069b500860d9 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:43.482523 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4684c457-2410-4062-a03e-4751e767e1cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f937d5e4b52646b49cea6b2d5b267069'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:43.483187 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4684c457-2410-4062-a03e-4751e767e1cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f937d5e4b52646b49cea6b2d5b267069'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:43.483623 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8163f-d6d1-4569-a078-069b500860d9 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:43.488586 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc3049-5c2d-4200-9079-50c93e1b7817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:43.489046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc3049-5c2d-4200-9079-50c93e1b7817 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:43.489323 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc3049-5c2d-4200-9079-50c93e1b7817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:43.489682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc3049-5c2d-4200-9079-50c93e1b7817 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:43.490643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc3049-5c2d-4200-9079-50c93e1b7817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:43.492486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc3049-5c2d-4200-9079-50c93e1b7817 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:43.495761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4684c457-2410-4062-a03e-4751e767e1cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f937d5e4b52646b49cea6b2d5b267069'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:43.496435 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4684c457-2410-4062-a03e-4751e767e1cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f937d5e4b52646b49cea6b2d5b267069'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:43.504283 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4684c457-2410-4062-a03e-4751e767e1cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f937d5e4b52646b49cea6b2d5b267069'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:43.573403 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 967/1922] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:43 2022] POST /identity/v3/projects => generated 375 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:17:43.577420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc3049-5c2d-4200-9079-50c93e1b7817 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:43.579178 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-5cbc3049-5c2d-4200-9079-50c93e1b7817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:43.601051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 956/1923] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:43 2022] DELETE /identity/v3/users/f937d5e4b52646b49cea6b2d5b267069 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:43.604514 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48edad1-5fa3-44a0-b416-814ad7dfb6d8 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:43.604861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48edad1-5fa3-44a0-b416-814ad7dfb6d8 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:43.605139 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48edad1-5fa3-44a0-b416-814ad7dfb6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:43.628513 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-b48edad1-5fa3-44a0-b416-814ad7dfb6d8 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:43.629786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b48edad1-5fa3-44a0-b416-814ad7dfb6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:43.665506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc3049-5c2d-4200-9079-50c93e1b7817 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:43.693872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 957/1924] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:43.703517 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48edad1-5fa3-44a0-b416-814ad7dfb6d8 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:43.704848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b48edad1-5fa3-44a0-b416-814ad7dfb6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:43.728371 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4640198-b7ca-4e70-a671-d8e381501d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:43.729045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4640198-b7ca-4e70-a671-d8e381501d48 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:43.729329 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4640198-b7ca-4e70-a671-d8e381501d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:43.729695 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4640198-b7ca-4e70-a671-d8e381501d48 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:43.730436 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4640198-b7ca-4e70-a671-d8e381501d48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:43.732386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4640198-b7ca-4e70-a671-d8e381501d48 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:43.760418 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48edad1-5fa3-44a0-b416-814ad7dfb6d8 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:43.764128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df6d70-fdb8-4eb2-8b93-faef0250dc92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:43.767891 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df6d70-fdb8-4eb2-8b93-faef0250dc92 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:43.767891 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df6d70-fdb8-4eb2-8b93-faef0250dc92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:43.767891 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df6d70-fdb8-4eb2-8b93-faef0250dc92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:43.774068 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df6d70-fdb8-4eb2-8b93-faef0250dc92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:43.775551 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df6d70-fdb8-4eb2-8b93-faef0250dc92 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:43.784788 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 958/1925] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:17:43.828665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 968/1926] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:43 2022] POST /identity/v3/users => generated 294 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:17:43.832317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df6d70-fdb8-4eb2-8b93-faef0250dc92 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:43.841689 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df6d70-fdb8-4eb2-8b93-faef0250dc92 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:43.846409 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178c46d-6657-4c28-bbe2-67bcea62f411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:43.847060 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178c46d-6657-4c28-bbe2-67bcea62f411 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:43.847342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178c46d-6657-4c28-bbe2-67bcea62f411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:43.847613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178c46d-6657-4c28-bbe2-67bcea62f411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:43.848329 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178c46d-6657-4c28-bbe2-67bcea62f411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:43.849910 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178c46d-6657-4c28-bbe2-67bcea62f411 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:43.860426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 959/1927] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:43.864643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4640198-b7ca-4e70-a671-d8e381501d48 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:43.874913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4640198-b7ca-4e70-a671-d8e381501d48 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:43.879540 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8357c49-caf1-4f50-8a66-b3f6ac245fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:43.883457 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8357c49-caf1-4f50-8a66-b3f6ac245fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:43.883457 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8357c49-caf1-4f50-8a66-b3f6ac245fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:43.883457 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8357c49-caf1-4f50-8a66-b3f6ac245fb1 admin admin] PATH_INFO: `/v3/projects/95df63b570d24cd4a277a3cd99440452/users/45e8fe7467834f9282775812813792c9/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:43.895021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8357c49-caf1-4f50-8a66-b3f6ac245fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95df63b570d24cd4a277a3cd99440452, user_id=45e8fe7467834f9282775812813792c9, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:43.897642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8357c49-caf1-4f50-8a66-b3f6ac245fb1 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:44.004546 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 969/1928] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:43 2022] PUT /identity/v3/projects/95df63b570d24cd4a277a3cd99440452/users/45e8fe7467834f9282775812813792c9/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:44.011606 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178c46d-6657-4c28-bbe2-67bcea62f411 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:44.026210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178c46d-6657-4c28-bbe2-67bcea62f411 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:44.095822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dcaa1-d87a-48c7-99c3-479fe8fc2a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:44.096745 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dcaa1-d87a-48c7-99c3-479fe8fc2a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:44.097141 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dcaa1-d87a-48c7-99c3-479fe8fc2a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:44.097568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dcaa1-d87a-48c7-99c3-479fe8fc2a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:44.098711 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6dcaa1-d87a-48c7-99c3-479fe8fc2a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:44.100545 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6dcaa1-d87a-48c7-99c3-479fe8fc2a41 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:44.116493 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 960/1929] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:44.121309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8357c49-caf1-4f50-8a66-b3f6ac245fb1 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:44.134040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8357c49-caf1-4f50-8a66-b3f6ac245fb1 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:44.148832 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5cf454-fe93-4434-a98c-fc6444bf8d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:44.148832 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5cf454-fe93-4434-a98c-fc6444bf8d57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:44.148832 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5cf454-fe93-4434-a98c-fc6444bf8d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:44.149391 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5cf454-fe93-4434-a98c-fc6444bf8d57 admin admin] PATH_INFO: `/v3/projects/95df63b570d24cd4a277a3cd99440452/users/45e8fe7467834f9282775812813792c9/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:44.156160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5cf454-fe93-4434-a98c-fc6444bf8d57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95df63b570d24cd4a277a3cd99440452, user_id=45e8fe7467834f9282775812813792c9, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:44.159601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5cf454-fe93-4434-a98c-fc6444bf8d57 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:44.171916 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 970/1930] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:44 2022] PUT /identity/v3/projects/95df63b570d24cd4a277a3cd99440452/users/45e8fe7467834f9282775812813792c9/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:44.183734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24281f-0b38-4e6c-9d84-56c3a0c5c30e None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:44.184025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24281f-0b38-4e6c-9d84-56c3a0c5c30e None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:44.184304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24281f-0b38-4e6c-9d84-56c3a0c5c30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:44.213838 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-0b24281f-0b38-4e6c-9d84-56c3a0c5c30e None None] MFA Rules not processed for user `45e8fe7467834f9282775812813792c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:44.215852 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0b24281f-0b38-4e6c-9d84-56c3a0c5c30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:44.322127 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 961/1931] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:44.330843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5cf454-fe93-4434-a98c-fc6444bf8d57 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:44.332565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-5a5cf454-fe93-4434-a98c-fc6444bf8d57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:44.390705 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5cf454-fe93-4434-a98c-fc6444bf8d57 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:44.392458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24281f-0b38-4e6c-9d84-56c3a0c5c30e None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:44.394412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0b24281f-0b38-4e6c-9d84-56c3a0c5c30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:44.394720 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f882d-034e-472d-accb-2654316f7b33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:44.395383 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f882d-034e-472d-accb-2654316f7b33 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:44.395669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f882d-034e-472d-accb-2654316f7b33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:44.395948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f882d-034e-472d-accb-2654316f7b33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:44.397807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-998f882d-034e-472d-accb-2654316f7b33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:44.470295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f882d-034e-472d-accb-2654316f7b33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:44.471748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f882d-034e-472d-accb-2654316f7b33 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:44.480925 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 971/1932] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:17:44.489135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24281f-0b38-4e6c-9d84-56c3a0c5c30e None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:44.494226 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05463ab6-ee93-48d2-b0d4-a7f5d6cd048e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:44.495144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05463ab6-ee93-48d2-b0d4-a7f5d6cd048e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:44.495529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05463ab6-ee93-48d2-b0d4-a7f5d6cd048e admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:44.495808 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05463ab6-ee93-48d2-b0d4-a7f5d6cd048e admin admin] PATH_INFO: `/v3/projects/48c1f68b49d44c62a39426915ec1b659` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:44.497016 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05463ab6-ee93-48d2-b0d4-a7f5d6cd048e admin admin] RBAC: Authorizing `identity:delete_project(project_id=48c1f68b49d44c62a39426915ec1b659)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:44.498955 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05463ab6-ee93-48d2-b0d4-a7f5d6cd048e admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:44.621111 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-05463ab6-ee93-48d2-b0d4-a7f5d6cd048e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48c1f68b49d44c62a39426915ec1b659', 'request_id': 'req-05463ab6-ee93-48d2-b0d4-a7f5d6cd048e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:44.725507 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-05463ab6-ee93-48d2-b0d4-a7f5d6cd048e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48c1f68b49d44c62a39426915ec1b659', 'request_id': 'req-05463ab6-ee93-48d2-b0d4-a7f5d6cd048e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:44.731833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-05463ab6-ee93-48d2-b0d4-a7f5d6cd048e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48c1f68b49d44c62a39426915ec1b659', 'request_id': 'req-05463ab6-ee93-48d2-b0d4-a7f5d6cd048e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:44.744385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-05463ab6-ee93-48d2-b0d4-a7f5d6cd048e admin admin] The token cache is being invalidate because project 48c1f68b49d44c62a39426915ec1b659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:44.744674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-05463ab6-ee93-48d2-b0d4-a7f5d6cd048e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:44.745352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-05463ab6-ee93-48d2-b0d4-a7f5d6cd048e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:44.747518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 962/1933] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:44 2022] DELETE /identity/v3/projects/48c1f68b49d44c62a39426915ec1b659 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:44.752054 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f882d-034e-472d-accb-2654316f7b33 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:44.763068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f882d-034e-472d-accb-2654316f7b33 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:44.826560 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8337b6-43ee-402b-b9e1-9f6287f0cb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:44.828241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8337b6-43ee-402b-b9e1-9f6287f0cb84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:44.828633 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8337b6-43ee-402b-b9e1-9f6287f0cb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:44.829052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8337b6-43ee-402b-b9e1-9f6287f0cb84 admin admin] PATH_INFO: `/v3/users/87dc297d6b9943abb45fd4a0d0b6791f` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:44.830454 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8337b6-43ee-402b-b9e1-9f6287f0cb84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87dc297d6b9943abb45fd4a0d0b6791f)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:44.832435 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8337b6-43ee-402b-b9e1-9f6287f0cb84 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:45.023174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6d8337b6-43ee-402b-b9e1-9f6287f0cb84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87dc297d6b9943abb45fd4a0d0b6791f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:45.023821 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6d8337b6-43ee-402b-b9e1-9f6287f0cb84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87dc297d6b9943abb45fd4a0d0b6791f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:45.024435 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6d8337b6-43ee-402b-b9e1-9f6287f0cb84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87dc297d6b9943abb45fd4a0d0b6791f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:45.036810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6d8337b6-43ee-402b-b9e1-9f6287f0cb84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87dc297d6b9943abb45fd4a0d0b6791f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:45.043762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6d8337b6-43ee-402b-b9e1-9f6287f0cb84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87dc297d6b9943abb45fd4a0d0b6791f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:45.130945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 972/1934] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:44 2022] DELETE /identity/v3/users/87dc297d6b9943abb45fd4a0d0b6791f => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:45.852288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05463ab6-ee93-48d2-b0d4-a7f5d6cd048e admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:45.852288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-05463ab6-ee93-48d2-b0d4-a7f5d6cd048e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:45.865477 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05463ab6-ee93-48d2-b0d4-a7f5d6cd048e admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:45.925065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a26f0-ea73-4dd2-9058-0912d8fe7548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:45.925932 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a26f0-ea73-4dd2-9058-0912d8fe7548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:45.926258 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a26f0-ea73-4dd2-9058-0912d8fe7548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:45.926537 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a26f0-ea73-4dd2-9058-0912d8fe7548 admin admin] PATH_INFO: `/v3/projects/1d577c7b91544e72a48eea267113c6c2` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:45.927867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a26f0-ea73-4dd2-9058-0912d8fe7548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d577c7b91544e72a48eea267113c6c2)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:45.929768 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a26f0-ea73-4dd2-9058-0912d8fe7548 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:46.057609 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-8a5a26f0-ea73-4dd2-9058-0912d8fe7548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d577c7b91544e72a48eea267113c6c2', 'request_id': 'req-8a5a26f0-ea73-4dd2-9058-0912d8fe7548'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:46.131282 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-8a5a26f0-ea73-4dd2-9058-0912d8fe7548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d577c7b91544e72a48eea267113c6c2', 'request_id': 'req-8a5a26f0-ea73-4dd2-9058-0912d8fe7548'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:46.137281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-8a5a26f0-ea73-4dd2-9058-0912d8fe7548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d577c7b91544e72a48eea267113c6c2', 'request_id': 'req-8a5a26f0-ea73-4dd2-9058-0912d8fe7548'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:46.148878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-8a5a26f0-ea73-4dd2-9058-0912d8fe7548 admin admin] The token cache is being invalidate because project 1d577c7b91544e72a48eea267113c6c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:46.149194 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-8a5a26f0-ea73-4dd2-9058-0912d8fe7548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:46.149825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-8a5a26f0-ea73-4dd2-9058-0912d8fe7548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:46.151372 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 963/1935] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:45 2022] DELETE /identity/v3/projects/1d577c7b91544e72a48eea267113c6c2 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:46.156691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3b0d1-a05e-4251-8bc6-7a5d3c0445e7 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:46.157142 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3b0d1-a05e-4251-8bc6-7a5d3c0445e7 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:46.157252 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3b0d1-a05e-4251-8bc6-7a5d3c0445e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:46.175908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-18a3b0d1-a05e-4251-8bc6-7a5d3c0445e7 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:46.177130 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-18a3b0d1-a05e-4251-8bc6-7a5d3c0445e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:46.237076 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 973/1936] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:46.244908 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c3491-eeae-445c-9283-3b2e41c12714 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:46.245194 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c3491-eeae-445c-9283-3b2e41c12714 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:46.245472 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c3491-eeae-445c-9283-3b2e41c12714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:46.263773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-c55c3491-eeae-445c-9283-3b2e41c12714 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:46.264878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c55c3491-eeae-445c-9283-3b2e41c12714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:46.273012 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 964/1937] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:46.277186 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3b0d1-a05e-4251-8bc6-7a5d3c0445e7 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:46.287864 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3b0d1-a05e-4251-8bc6-7a5d3c0445e7 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:46.292572 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bd407-44dc-4295-ad09-e77deb9068e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:46.293255 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bd407-44dc-4295-ad09-e77deb9068e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:46.293541 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bd407-44dc-4295-ad09-e77deb9068e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:46.293818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bd407-44dc-4295-ad09-e77deb9068e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:46.294591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bd407-44dc-4295-ad09-e77deb9068e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:17:46.294903 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bd407-44dc-4295-ad09-e77deb9068e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:46.297141 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bd407-44dc-4295-ad09-e77deb9068e5 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:46.308904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 974/1938] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:17:46.316965 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c3491-eeae-445c-9283-3b2e41c12714 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:46.326746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c3491-eeae-445c-9283-3b2e41c12714 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:46.332970 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13833906-b555-4d14-afc5-967dfa3c0035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:46.333622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13833906-b555-4d14-afc5-967dfa3c0035 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:46.334052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13833906-b555-4d14-afc5-967dfa3c0035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:46.334356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13833906-b555-4d14-afc5-967dfa3c0035 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:46.335082 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13833906-b555-4d14-afc5-967dfa3c0035 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:46.337014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13833906-b555-4d14-afc5-967dfa3c0035 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:46.418324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 965/1939] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:46 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:46.424434 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bd407-44dc-4295-ad09-e77deb9068e5 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:46.433396 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bd407-44dc-4295-ad09-e77deb9068e5 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:46.491119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e1bb4-e9bd-40c5-9026-20a437570e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:46.491797 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e1bb4-e9bd-40c5-9026-20a437570e1a admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:46.492084 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e1bb4-e9bd-40c5-9026-20a437570e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:46.492369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e1bb4-e9bd-40c5-9026-20a437570e1a admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:46.493075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e1bb4-e9bd-40c5-9026-20a437570e1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:46.495099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e1bb4-e9bd-40c5-9026-20a437570e1a admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:46.576629 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 975/1940] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:46 2022] POST /identity/v3/users => generated 285 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:46.580453 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13833906-b555-4d14-afc5-967dfa3c0035 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:46.590730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13833906-b555-4d14-afc5-967dfa3c0035 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:46.595357 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0fee2-5d0c-4019-8409-0ea85efd0173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:46.596003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0fee2-5d0c-4019-8409-0ea85efd0173 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:46.596285 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0fee2-5d0c-4019-8409-0ea85efd0173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:46.596558 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0fee2-5d0c-4019-8409-0ea85efd0173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:46.597277 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0fee2-5d0c-4019-8409-0ea85efd0173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:46.598931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0fee2-5d0c-4019-8409-0ea85efd0173 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:46.609488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 966/1941] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:46.613303 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e1bb4-e9bd-40c5-9026-20a437570e1a admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:46.623143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e1bb4-e9bd-40c5-9026-20a437570e1a admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:46.629237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c25180-0eb8-4f7f-9057-ec03cb5f6c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:46.629929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c25180-0eb8-4f7f-9057-ec03cb5f6c45 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:46.630246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c25180-0eb8-4f7f-9057-ec03cb5f6c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:46.630524 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c25180-0eb8-4f7f-9057-ec03cb5f6c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:46.634200 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c25180-0eb8-4f7f-9057-ec03cb5f6c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:46.634200 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c25180-0eb8-4f7f-9057-ec03cb5f6c45 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:46.644615 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 976/1942] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:46.648921 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0fee2-5d0c-4019-8409-0ea85efd0173 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:46.665157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0fee2-5d0c-4019-8409-0ea85efd0173 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:46.672222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f52275-36e7-451c-b4ed-725f3d8e6fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:46.673054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f52275-36e7-451c-b4ed-725f3d8e6fd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:46.673336 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f52275-36e7-451c-b4ed-725f3d8e6fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:46.673611 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f52275-36e7-451c-b4ed-725f3d8e6fd4 admin admin] PATH_INFO: `/v3/projects/6ddb856d21a1486bb58ca2429d5d6249/users/b61ed48eba0d4709878e73629548b3bd/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:46.692355 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f52275-36e7-451c-b4ed-725f3d8e6fd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ddb856d21a1486bb58ca2429d5d6249, user_id=b61ed48eba0d4709878e73629548b3bd, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:46.695021 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f52275-36e7-451c-b4ed-725f3d8e6fd4 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:46.776735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 967/1943] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:46 2022] PUT /identity/v3/projects/6ddb856d21a1486bb58ca2429d5d6249/users/b61ed48eba0d4709878e73629548b3bd/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:46.787738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903e98a-9e03-4828-9a4c-7a649dbc18c1 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:46.788028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903e98a-9e03-4828-9a4c-7a649dbc18c1 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:46.788308 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903e98a-9e03-4828-9a4c-7a649dbc18c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:46.816322 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-2903e98a-9e03-4828-9a4c-7a649dbc18c1 None None] MFA Rules not processed for user `b61ed48eba0d4709878e73629548b3bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:46.817537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-2903e98a-9e03-4828-9a4c-7a649dbc18c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:46.891745 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 977/1944] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:46 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:46.896094 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f52275-36e7-451c-b4ed-725f3d8e6fd4 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:46.957816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f52275-36e7-451c-b4ed-725f3d8e6fd4 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:46.962455 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d980e-3900-4c7e-b25a-900f2b5c53b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:46.963103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d980e-3900-4c7e-b25a-900f2b5c53b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:46.963386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d980e-3900-4c7e-b25a-900f2b5c53b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:46.963656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d980e-3900-4c7e-b25a-900f2b5c53b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:46.964345 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d980e-3900-4c7e-b25a-900f2b5c53b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:46.966255 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d980e-3900-4c7e-b25a-900f2b5c53b3 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:47.065158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 968/1945] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:46 2022] POST /identity/v3/projects => generated 369 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:47.068968 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903e98a-9e03-4828-9a4c-7a649dbc18c1 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:47.080099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903e98a-9e03-4828-9a4c-7a649dbc18c1 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:47.133678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e53397-cc39-495e-9649-653732cb0425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:47.134381 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e53397-cc39-495e-9649-653732cb0425 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:47.134666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e53397-cc39-495e-9649-653732cb0425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:47.134942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e53397-cc39-495e-9649-653732cb0425 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:47.135649 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e53397-cc39-495e-9649-653732cb0425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:47.137549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e53397-cc39-495e-9649-653732cb0425 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:47.217042 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 978/1946] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:47 2022] POST /identity/v3/users => generated 291 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:47.220791 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d980e-3900-4c7e-b25a-900f2b5c53b3 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:47.231021 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d980e-3900-4c7e-b25a-900f2b5c53b3 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:47.235885 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2606a9-f0de-49bd-9afc-91efc9e5aaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:47.236552 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2606a9-f0de-49bd-9afc-91efc9e5aaec admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:47.236863 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2606a9-f0de-49bd-9afc-91efc9e5aaec admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:47.237142 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2606a9-f0de-49bd-9afc-91efc9e5aaec admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:47.237881 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2606a9-f0de-49bd-9afc-91efc9e5aaec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:47.239695 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2606a9-f0de-49bd-9afc-91efc9e5aaec admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:47.251035 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 969/1947] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:47.255053 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e53397-cc39-495e-9649-653732cb0425 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:47.264948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e53397-cc39-495e-9649-653732cb0425 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:47.270205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324d16f-955f-4d99-b56c-20365b80747b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:47.271108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324d16f-955f-4d99-b56c-20365b80747b admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:47.271393 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324d16f-955f-4d99-b56c-20365b80747b admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:47.271673 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324d16f-955f-4d99-b56c-20365b80747b admin admin] PATH_INFO: `/v3/projects/6e3795bddd3b4557a8d8e0004ac54aed/users/31f15534342a4f3294d25b5db326b2cd/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:47.287193 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324d16f-955f-4d99-b56c-20365b80747b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e3795bddd3b4557a8d8e0004ac54aed, user_id=31f15534342a4f3294d25b5db326b2cd, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:47.289824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324d16f-955f-4d99-b56c-20365b80747b admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:47.365505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 979/1948] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:47 2022] PUT /identity/v3/projects/6e3795bddd3b4557a8d8e0004ac54aed/users/31f15534342a4f3294d25b5db326b2cd/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:47.369404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2606a9-f0de-49bd-9afc-91efc9e5aaec admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:47.379940 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2606a9-f0de-49bd-9afc-91efc9e5aaec admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:47.434435 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7638a1b-7288-4a50-ae14-8f752f2c3f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:47.435110 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7638a1b-7288-4a50-ae14-8f752f2c3f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:47.435389 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7638a1b-7288-4a50-ae14-8f752f2c3f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:47.435660 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7638a1b-7288-4a50-ae14-8f752f2c3f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:47.436416 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7638a1b-7288-4a50-ae14-8f752f2c3f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:47.438706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7638a1b-7288-4a50-ae14-8f752f2c3f58 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:47.448479 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 970/1949] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:47.452599 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324d16f-955f-4d99-b56c-20365b80747b admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:47.462603 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324d16f-955f-4d99-b56c-20365b80747b admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:47.467113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef29fe0-89e4-49b9-9ffa-ca4ab7bfd93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:47.467916 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef29fe0-89e4-49b9-9ffa-ca4ab7bfd93d admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:47.468194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef29fe0-89e4-49b9-9ffa-ca4ab7bfd93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:47.468477 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef29fe0-89e4-49b9-9ffa-ca4ab7bfd93d admin admin] PATH_INFO: `/v3/projects/6e3795bddd3b4557a8d8e0004ac54aed/users/31f15534342a4f3294d25b5db326b2cd/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:47.470431 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef29fe0-89e4-49b9-9ffa-ca4ab7bfd93d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e3795bddd3b4557a8d8e0004ac54aed, user_id=31f15534342a4f3294d25b5db326b2cd, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:47.473025 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef29fe0-89e4-49b9-9ffa-ca4ab7bfd93d admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:47.482698 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 980/1950] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:47 2022] PUT /identity/v3/projects/6e3795bddd3b4557a8d8e0004ac54aed/users/31f15534342a4f3294d25b5db326b2cd/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:47.494836 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826b3f8-dc19-43a8-9c7e-8fcf9de88bd6 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:47.495129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826b3f8-dc19-43a8-9c7e-8fcf9de88bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:47.495410 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826b3f8-dc19-43a8-9c7e-8fcf9de88bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:47.521534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-d826b3f8-dc19-43a8-9c7e-8fcf9de88bd6 None None] MFA Rules not processed for user `31f15534342a4f3294d25b5db326b2cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:47.522765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-d826b3f8-dc19-43a8-9c7e-8fcf9de88bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:47.590238 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 971/1951] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:47 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:47.600758 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef29fe0-89e4-49b9-9ffa-ca4ab7bfd93d admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:47.602460 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-8ef29fe0-89e4-49b9-9ffa-ca4ab7bfd93d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:47.657726 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef29fe0-89e4-49b9-9ffa-ca4ab7bfd93d admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:47.661395 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd3e06-b2ce-4f89-a3bd-197f950ffa8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:47.662035 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd3e06-b2ce-4f89-a3bd-197f950ffa8b service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:47.662346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd3e06-b2ce-4f89-a3bd-197f950ffa8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:47.662621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd3e06-b2ce-4f89-a3bd-197f950ffa8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:47.672143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd3e06-b2ce-4f89-a3bd-197f950ffa8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:47.673633 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd3e06-b2ce-4f89-a3bd-197f950ffa8b service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:47.683219 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 981/1952] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:47 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:17:50.391643 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826b3f8-dc19-43a8-9c7e-8fcf9de88bd6 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:50.458878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826b3f8-dc19-43a8-9c7e-8fcf9de88bd6 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:50.468171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89826531-330e-4586-817f-a211c8e8db64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:50.468171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89826531-330e-4586-817f-a211c8e8db64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:50.468171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89826531-330e-4586-817f-a211c8e8db64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:50.468171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89826531-330e-4586-817f-a211c8e8db64 admin admin] PATH_INFO: `/v3/users/b61ed48eba0d4709878e73629548b3bd` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:50.468597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89826531-330e-4586-817f-a211c8e8db64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b61ed48eba0d4709878e73629548b3bd)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:50.469288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89826531-330e-4586-817f-a211c8e8db64 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:50.689506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd3e06-b2ce-4f89-a3bd-197f950ffa8b service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:50.691750 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-67fd3e06-b2ce-4f89-a3bd-197f950ffa8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:50.709835 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd3e06-b2ce-4f89-a3bd-197f950ffa8b service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:50.717042 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-89826531-330e-4586-817f-a211c8e8db64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b61ed48eba0d4709878e73629548b3bd'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:50.717389 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fefe99-b03c-45ca-806e-bcda7bad73f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:50.718108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-89826531-330e-4586-817f-a211c8e8db64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b61ed48eba0d4709878e73629548b3bd'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:50.718266 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fefe99-b03c-45ca-806e-bcda7bad73f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:50.718656 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fefe99-b03c-45ca-806e-bcda7bad73f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:50.719627 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fefe99-b03c-45ca-806e-bcda7bad73f5 admin admin] PATH_INFO: `/v3/users/1b5de9557e2341fe9eb05fc3f17bf620` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:50.725045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fefe99-b03c-45ca-806e-bcda7bad73f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b5de9557e2341fe9eb05fc3f17bf620)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:50.725045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fefe99-b03c-45ca-806e-bcda7bad73f5 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:50.746364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-89826531-330e-4586-817f-a211c8e8db64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b61ed48eba0d4709878e73629548b3bd'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:50.747264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-89826531-330e-4586-817f-a211c8e8db64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b61ed48eba0d4709878e73629548b3bd'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:50.756633 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-89826531-330e-4586-817f-a211c8e8db64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b61ed48eba0d4709878e73629548b3bd'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:50.861780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 972/1953] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:50 2022] DELETE /identity/v3/users/b61ed48eba0d4709878e73629548b3bd => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:50.866867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6764d-6b02-40fd-aac7-61ec2205341a None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:50.867239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6764d-6b02-40fd-aac7-61ec2205341a None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:50.867589 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6764d-6b02-40fd-aac7-61ec2205341a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:50.887133 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-c8f6764d-6b02-40fd-aac7-61ec2205341a None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:50.887931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c8f6764d-6b02-40fd-aac7-61ec2205341a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:50.954269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-95fefe99-b03c-45ca-806e-bcda7bad73f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b5de9557e2341fe9eb05fc3f17bf620'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:50.956095 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-95fefe99-b03c-45ca-806e-bcda7bad73f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b5de9557e2341fe9eb05fc3f17bf620'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:50.957819 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-95fefe99-b03c-45ca-806e-bcda7bad73f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b5de9557e2341fe9eb05fc3f17bf620'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:50.977109 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 973/1954] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:50.985997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-95fefe99-b03c-45ca-806e-bcda7bad73f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b5de9557e2341fe9eb05fc3f17bf620'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:50.990236 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6764d-6b02-40fd-aac7-61ec2205341a None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:50.992001 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c8f6764d-6b02-40fd-aac7-61ec2205341a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:50.995929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-95fefe99-b03c-45ca-806e-bcda7bad73f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b5de9557e2341fe9eb05fc3f17bf620'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:51.063789 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6764d-6b02-40fd-aac7-61ec2205341a None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:51.069369 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125bf26-d851-4cd7-9257-6f77ccfd7da2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:51.070041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125bf26-d851-4cd7-9257-6f77ccfd7da2 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:51.073185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125bf26-d851-4cd7-9257-6f77ccfd7da2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:51.073185 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125bf26-d851-4cd7-9257-6f77ccfd7da2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:51.081586 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125bf26-d851-4cd7-9257-6f77ccfd7da2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:51.083076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125bf26-d851-4cd7-9257-6f77ccfd7da2 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:51.093528 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 974/1955] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:17:51.105224 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 982/1956] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:50 2022] DELETE /identity/v3/users/1b5de9557e2341fe9eb05fc3f17bf620 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:51.111737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1974cfa-941a-4dea-a970-5ad4385707c7 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:51.112029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1974cfa-941a-4dea-a970-5ad4385707c7 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:51.112299 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1974cfa-941a-4dea-a970-5ad4385707c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:51.133029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-e1974cfa-941a-4dea-a970-5ad4385707c7 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:51.134440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-e1974cfa-941a-4dea-a970-5ad4385707c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:51.221255 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 975/1957] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:51.236810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fefe99-b03c-45ca-806e-bcda7bad73f5 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:51.238317 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-95fefe99-b03c-45ca-806e-bcda7bad73f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:51.302865 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fefe99-b03c-45ca-806e-bcda7bad73f5 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:51.306671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576465a-9c04-4270-a579-e28d11f58f41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:51.307316 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576465a-9c04-4270-a579-e28d11f58f41 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:51.307600 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576465a-9c04-4270-a579-e28d11f58f41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:51.307874 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576465a-9c04-4270-a579-e28d11f58f41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:51.317119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576465a-9c04-4270-a579-e28d11f58f41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:51.318591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576465a-9c04-4270-a579-e28d11f58f41 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:51.333888 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 983/1958] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:17:51.580828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1974cfa-941a-4dea-a970-5ad4385707c7 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:51.583234 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-e1974cfa-941a-4dea-a970-5ad4385707c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:51.627510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1974cfa-941a-4dea-a970-5ad4385707c7 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:51.633516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afb941-244f-403a-a364-dc27a6bbf9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:51.634846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afb941-244f-403a-a364-dc27a6bbf9e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:51.635294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afb941-244f-403a-a364-dc27a6bbf9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:51.635650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afb941-244f-403a-a364-dc27a6bbf9e1 admin admin] PATH_INFO: `/v3/projects/6ddb856d21a1486bb58ca2429d5d6249` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:51.637228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afb941-244f-403a-a364-dc27a6bbf9e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ddb856d21a1486bb58ca2429d5d6249)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:51.639850 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afb941-244f-403a-a364-dc27a6bbf9e1 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:51.809049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-b2afb941-244f-403a-a364-dc27a6bbf9e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ddb856d21a1486bb58ca2429d5d6249', 'request_id': 'req-b2afb941-244f-403a-a364-dc27a6bbf9e1'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:51.875218 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-b2afb941-244f-403a-a364-dc27a6bbf9e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ddb856d21a1486bb58ca2429d5d6249', 'request_id': 'req-b2afb941-244f-403a-a364-dc27a6bbf9e1'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:51.883069 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-b2afb941-244f-403a-a364-dc27a6bbf9e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ddb856d21a1486bb58ca2429d5d6249', 'request_id': 'req-b2afb941-244f-403a-a364-dc27a6bbf9e1'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:51.895276 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-b2afb941-244f-403a-a364-dc27a6bbf9e1 admin admin] The token cache is being invalidate because project 6ddb856d21a1486bb58ca2429d5d6249 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:51.895566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-b2afb941-244f-403a-a364-dc27a6bbf9e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:51.896239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-b2afb941-244f-403a-a364-dc27a6bbf9e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:51.897797 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 976/1959] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:51 2022] DELETE /identity/v3/projects/6ddb856d21a1486bb58ca2429d5d6249 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:51.901705 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576465a-9c04-4270-a579-e28d11f58f41 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:51.911557 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576465a-9c04-4270-a579-e28d11f58f41 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:51.967911 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da148f5-daa9-4953-bc98-13110611bf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:51.968781 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da148f5-daa9-4953-bc98-13110611bf1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:51.969107 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da148f5-daa9-4953-bc98-13110611bf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:51.969387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da148f5-daa9-4953-bc98-13110611bf1e admin admin] PATH_INFO: `/v3/users/31f15534342a4f3294d25b5db326b2cd` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:51.970655 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da148f5-daa9-4953-bc98-13110611bf1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=31f15534342a4f3294d25b5db326b2cd)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:51.972543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da148f5-daa9-4953-bc98-13110611bf1e admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:52.081182 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afb941-244f-403a-a364-dc27a6bbf9e1 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:52.082894 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b2afb941-244f-403a-a364-dc27a6bbf9e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:52.096768 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afb941-244f-403a-a364-dc27a6bbf9e1 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:52.102040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4bbca-973a-41a6-b6bd-debe309bea02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:52.102998 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4bbca-973a-41a6-b6bd-debe309bea02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:52.103217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4bbca-973a-41a6-b6bd-debe309bea02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:52.103491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4bbca-973a-41a6-b6bd-debe309bea02 admin admin] PATH_INFO: `/v3/projects/a9d63255507b45d285978da72081888a` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:52.104780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c4bbca-973a-41a6-b6bd-debe309bea02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9d63255507b45d285978da72081888a)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:52.106640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c4bbca-973a-41a6-b6bd-debe309bea02 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:52.198022 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7da148f5-daa9-4953-bc98-13110611bf1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31f15534342a4f3294d25b5db326b2cd'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:52.198785 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7da148f5-daa9-4953-bc98-13110611bf1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31f15534342a4f3294d25b5db326b2cd'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:52.199581 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7da148f5-daa9-4953-bc98-13110611bf1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31f15534342a4f3294d25b5db326b2cd'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:52.211307 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7da148f5-daa9-4953-bc98-13110611bf1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31f15534342a4f3294d25b5db326b2cd'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:52.218052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7da148f5-daa9-4953-bc98-13110611bf1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31f15534342a4f3294d25b5db326b2cd'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:52.257688 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-88c4bbca-973a-41a6-b6bd-debe309bea02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9d63255507b45d285978da72081888a', 'request_id': 'req-88c4bbca-973a-41a6-b6bd-debe309bea02'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:52.331944 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 984/1960] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:51 2022] DELETE /identity/v3/users/31f15534342a4f3294d25b5db326b2cd => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:52.383496 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-88c4bbca-973a-41a6-b6bd-debe309bea02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9d63255507b45d285978da72081888a', 'request_id': 'req-88c4bbca-973a-41a6-b6bd-debe309bea02'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:52.396499 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-88c4bbca-973a-41a6-b6bd-debe309bea02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9d63255507b45d285978da72081888a', 'request_id': 'req-88c4bbca-973a-41a6-b6bd-debe309bea02'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:52.406265 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-88c4bbca-973a-41a6-b6bd-debe309bea02 admin admin] The token cache is being invalidate because project a9d63255507b45d285978da72081888a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:52.407138 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-88c4bbca-973a-41a6-b6bd-debe309bea02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:52.407813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-88c4bbca-973a-41a6-b6bd-debe309bea02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:52.409640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 977/1961] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:52 2022] DELETE /identity/v3/projects/a9d63255507b45d285978da72081888a => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:52.413382 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da148f5-daa9-4953-bc98-13110611bf1e admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:52.414891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7da148f5-daa9-4953-bc98-13110611bf1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:52.479918 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da148f5-daa9-4953-bc98-13110611bf1e admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:52.484715 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5521a-1c71-4549-8c91-d9c48689d9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:52.486103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5521a-1c71-4549-8c91-d9c48689d9e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:52.486399 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5521a-1c71-4549-8c91-d9c48689d9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:52.486685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5521a-1c71-4549-8c91-d9c48689d9e6 admin admin] PATH_INFO: `/v3/users/45e8fe7467834f9282775812813792c9` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:52.487890 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5521a-1c71-4549-8c91-d9c48689d9e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45e8fe7467834f9282775812813792c9)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:52.489810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5521a-1c71-4549-8c91-d9c48689d9e6 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:52.719841 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-50c5521a-1c71-4549-8c91-d9c48689d9e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45e8fe7467834f9282775812813792c9'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:52.720501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-50c5521a-1c71-4549-8c91-d9c48689d9e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45e8fe7467834f9282775812813792c9'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:52.721125 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-50c5521a-1c71-4549-8c91-d9c48689d9e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45e8fe7467834f9282775812813792c9'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:52.744558 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-50c5521a-1c71-4549-8c91-d9c48689d9e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45e8fe7467834f9282775812813792c9'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:52.751537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-50c5521a-1c71-4549-8c91-d9c48689d9e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45e8fe7467834f9282775812813792c9'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:52.772997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4bbca-973a-41a6-b6bd-debe309bea02 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:52.775088 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-88c4bbca-973a-41a6-b6bd-debe309bea02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:52.861635 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 985/1962] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:52 2022] DELETE /identity/v3/users/45e8fe7467834f9282775812813792c9 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:52.873560 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4bbca-973a-41a6-b6bd-debe309bea02 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:52.995814 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca473e0-2c61-4656-809b-5fc76a42b78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:52.996760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca473e0-2c61-4656-809b-5fc76a42b78d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:52.999329 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca473e0-2c61-4656-809b-5fc76a42b78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:52.999329 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca473e0-2c61-4656-809b-5fc76a42b78d admin admin] PATH_INFO: `/v3/projects/6e3795bddd3b4557a8d8e0004ac54aed` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:52.999329 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca473e0-2c61-4656-809b-5fc76a42b78d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e3795bddd3b4557a8d8e0004ac54aed)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:53.001220 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca473e0-2c61-4656-809b-5fc76a42b78d admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:53.150462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1ca473e0-2c61-4656-809b-5fc76a42b78d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e3795bddd3b4557a8d8e0004ac54aed', 'request_id': 'req-1ca473e0-2c61-4656-809b-5fc76a42b78d'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:53.223893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1ca473e0-2c61-4656-809b-5fc76a42b78d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e3795bddd3b4557a8d8e0004ac54aed', 'request_id': 'req-1ca473e0-2c61-4656-809b-5fc76a42b78d'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:53.230149 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1ca473e0-2c61-4656-809b-5fc76a42b78d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e3795bddd3b4557a8d8e0004ac54aed', 'request_id': 'req-1ca473e0-2c61-4656-809b-5fc76a42b78d'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:53.241143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1ca473e0-2c61-4656-809b-5fc76a42b78d admin admin] The token cache is being invalidate because project 6e3795bddd3b4557a8d8e0004ac54aed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:53.241466 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1ca473e0-2c61-4656-809b-5fc76a42b78d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:53.242130 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1ca473e0-2c61-4656-809b-5fc76a42b78d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:53.243677 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 978/1963] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:52 2022] DELETE /identity/v3/projects/6e3795bddd3b4557a8d8e0004ac54aed => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:53.248491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56705bce-891e-4a1e-b9f2-8e1de3a591ca None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:53.248938 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56705bce-891e-4a1e-b9f2-8e1de3a591ca None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:53.249220 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56705bce-891e-4a1e-b9f2-8e1de3a591ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:53.267437 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-56705bce-891e-4a1e-b9f2-8e1de3a591ca None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:53.268634 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-56705bce-891e-4a1e-b9f2-8e1de3a591ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:53.327923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 986/1964] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:53.336606 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724644b-4157-4297-8aa3-0ff6232feb6c None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:53.336894 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724644b-4157-4297-8aa3-0ff6232feb6c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:53.337169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724644b-4157-4297-8aa3-0ff6232feb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:53.356630 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-8724644b-4157-4297-8aa3-0ff6232feb6c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:53.357822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8724644b-4157-4297-8aa3-0ff6232feb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:53.367283 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 979/1965] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:53.371814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56705bce-891e-4a1e-b9f2-8e1de3a591ca None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:53.374198 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724644b-4157-4297-8aa3-0ff6232feb6c None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:53.375978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8724644b-4157-4297-8aa3-0ff6232feb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:53.384897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56705bce-891e-4a1e-b9f2-8e1de3a591ca None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:53.388363 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724644b-4157-4297-8aa3-0ff6232feb6c None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:53.391585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194d445-3ae0-4a2e-9536-7c592bd8497a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:53.392923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194d445-3ae0-4a2e-9536-7c592bd8497a admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:53.393346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194d445-3ae0-4a2e-9536-7c592bd8497a admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:53.393732 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194d445-3ae0-4a2e-9536-7c592bd8497a admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:53.393843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbc5ee-d489-424f-85cf-d54434c6933b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:53.394684 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbc5ee-d489-424f-85cf-d54434c6933b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:53.394988 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2194d445-3ae0-4a2e-9536-7c592bd8497a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:17:53.395097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbc5ee-d489-424f-85cf-d54434c6933b admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:53.395257 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbc5ee-d489-424f-85cf-d54434c6933b admin admin] PATH_INFO: `/v3/projects/95df63b570d24cd4a277a3cd99440452` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:53.395645 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2194d445-3ae0-4a2e-9536-7c592bd8497a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:53.396565 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbc5ee-d489-424f-85cf-d54434c6933b admin admin] RBAC: Authorizing `identity:delete_project(project_id=95df63b570d24cd4a277a3cd99440452)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:53.398491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbc5ee-d489-424f-85cf-d54434c6933b admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:53.398719 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2194d445-3ae0-4a2e-9536-7c592bd8497a admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:53.412166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 987/1966] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 20:17:53.416597 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194d445-3ae0-4a2e-9536-7c592bd8497a admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:53.432852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194d445-3ae0-4a2e-9536-7c592bd8497a admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:53.437708 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e542c7-1970-4725-91b6-c9b729fe71f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:53.439083 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e542c7-1970-4725-91b6-c9b729fe71f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:53.439379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e542c7-1970-4725-91b6-c9b729fe71f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:53.439663 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e542c7-1970-4725-91b6-c9b729fe71f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:53.440376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e542c7-1970-4725-91b6-c9b729fe71f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:53.442344 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e542c7-1970-4725-91b6-c9b729fe71f5 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:53.497800 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3fdbc5ee-d489-424f-85cf-d54434c6933b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95df63b570d24cd4a277a3cd99440452', 'request_id': 'req-3fdbc5ee-d489-424f-85cf-d54434c6933b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:53.572216 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 988/1967] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:53 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:17:53.576215 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e542c7-1970-4725-91b6-c9b729fe71f5 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:53.593093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3fdbc5ee-d489-424f-85cf-d54434c6933b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95df63b570d24cd4a277a3cd99440452', 'request_id': 'req-3fdbc5ee-d489-424f-85cf-d54434c6933b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:53.598967 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3fdbc5ee-d489-424f-85cf-d54434c6933b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95df63b570d24cd4a277a3cd99440452', 'request_id': 'req-3fdbc5ee-d489-424f-85cf-d54434c6933b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:53.610958 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3fdbc5ee-d489-424f-85cf-d54434c6933b admin admin] The token cache is being invalidate because project 95df63b570d24cd4a277a3cd99440452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:53.611246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3fdbc5ee-d489-424f-85cf-d54434c6933b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:53.611899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3fdbc5ee-d489-424f-85cf-d54434c6933b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:53.613445 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 980/1968] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:53 2022] DELETE /identity/v3/projects/95df63b570d24cd4a277a3cd99440452 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:53.618208 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88e9de-1315-4588-a6e1-cfb915cb6462 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:53.618498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88e9de-1315-4588-a6e1-cfb915cb6462 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:53.618776 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88e9de-1315-4588-a6e1-cfb915cb6462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:53.636941 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e542c7-1970-4725-91b6-c9b729fe71f5 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:53.637412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-bb88e9de-1315-4588-a6e1-cfb915cb6462 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:53.638637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-bb88e9de-1315-4588-a6e1-cfb915cb6462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:53.641662 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816a6c1-97ca-4bac-9176-a06d9c7a98fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:53.642350 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816a6c1-97ca-4bac-9176-a06d9c7a98fc admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:53.642631 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816a6c1-97ca-4bac-9176-a06d9c7a98fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:53.642904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816a6c1-97ca-4bac-9176-a06d9c7a98fc admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:53.643645 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816a6c1-97ca-4bac-9176-a06d9c7a98fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:53.645530 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816a6c1-97ca-4bac-9176-a06d9c7a98fc admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:53.646594 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 981/1969] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:53.655766 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77304170-5569-4abd-b7c2-5e84ad166a35 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:53.656036 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77304170-5569-4abd-b7c2-5e84ad166a35 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:53.656312 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77304170-5569-4abd-b7c2-5e84ad166a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:53.675379 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-77304170-5569-4abd-b7c2-5e84ad166a35 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:53.676566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-77304170-5569-4abd-b7c2-5e84ad166a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:53.684839 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 982/1970] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:53.689487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77304170-5569-4abd-b7c2-5e84ad166a35 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:53.698656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77304170-5569-4abd-b7c2-5e84ad166a35 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:53.703613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68ab56-9e1e-4b59-be01-97f6ec7dca67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:53.704862 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68ab56-9e1e-4b59-be01-97f6ec7dca67 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:53.705149 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68ab56-9e1e-4b59-be01-97f6ec7dca67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:53.706416 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68ab56-9e1e-4b59-be01-97f6ec7dca67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:53.706416 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd68ab56-9e1e-4b59-be01-97f6ec7dca67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:17:53.706563 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd68ab56-9e1e-4b59-be01-97f6ec7dca67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:53.708101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd68ab56-9e1e-4b59-be01-97f6ec7dca67 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:53.719753 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 983/1971] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 20:17:53.723879 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68ab56-9e1e-4b59-be01-97f6ec7dca67 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:53.730685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 989/1972] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:53 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:17:53.734694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68ab56-9e1e-4b59-be01-97f6ec7dca67 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:53.739637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f2061-787c-4935-b8a7-4ca3af12f545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:53.740177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816a6c1-97ca-4bac-9176-a06d9c7a98fc admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:53.740308 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f2061-787c-4935-b8a7-4ca3af12f545 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:53.740542 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f2061-787c-4935-b8a7-4ca3af12f545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:53.740818 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f2061-787c-4935-b8a7-4ca3af12f545 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:53.741501 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f2061-787c-4935-b8a7-4ca3af12f545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:53.741806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-9816a6c1-97ca-4bac-9176-a06d9c7a98fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:53.743484 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f2061-787c-4935-b8a7-4ca3af12f545 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:53.753988 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816a6c1-97ca-4bac-9176-a06d9c7a98fc admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:53.758839 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5081c1-c27c-4554-a9e6-3a481e68c252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:53.759504 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5081c1-c27c-4554-a9e6-3a481e68c252 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:53.759802 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5081c1-c27c-4554-a9e6-3a481e68c252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:53.760080 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5081c1-c27c-4554-a9e6-3a481e68c252 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:53.760864 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5081c1-c27c-4554-a9e6-3a481e68c252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:53.762585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5081c1-c27c-4554-a9e6-3a481e68c252 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:53.772947 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 990/1973] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:17:53.776750 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5081c1-c27c-4554-a9e6-3a481e68c252 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:53.786195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5081c1-c27c-4554-a9e6-3a481e68c252 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:53.790980 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01cd7df-51a6-4add-8563-f737a9bed803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:53.791604 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01cd7df-51a6-4add-8563-f737a9bed803 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:53.791892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01cd7df-51a6-4add-8563-f737a9bed803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:53.792173 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01cd7df-51a6-4add-8563-f737a9bed803 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:53.792883 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01cd7df-51a6-4add-8563-f737a9bed803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:53.794475 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01cd7df-51a6-4add-8563-f737a9bed803 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:53.805392 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 991/1974] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:17:53.809457 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01cd7df-51a6-4add-8563-f737a9bed803 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:53.819287 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01cd7df-51a6-4add-8563-f737a9bed803 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:53.823923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5be5f-b7bc-4c05-bdab-d5c7510f624f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:53.824788 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5be5f-b7bc-4c05-bdab-d5c7510f624f admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:53.825071 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5be5f-b7bc-4c05-bdab-d5c7510f624f admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:53.825343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5be5f-b7bc-4c05-bdab-d5c7510f624f admin admin] PATH_INFO: `/v3/projects/64e4263d76ee40548fec18021cee5b10/users/504e7605c1f741609bae7c70d4d62875/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:53.841135 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e5be5f-b7bc-4c05-bdab-d5c7510f624f admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e4263d76ee40548fec18021cee5b10, user_id=504e7605c1f741609bae7c70d4d62875, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:53.843954 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e5be5f-b7bc-4c05-bdab-d5c7510f624f admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:53.865060 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 984/1975] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:53 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:17:53.869263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f2061-787c-4935-b8a7-4ca3af12f545 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:53.926976 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 992/1976] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:53 2022] PUT /identity/v3/projects/64e4263d76ee40548fec18021cee5b10/users/504e7605c1f741609bae7c70d4d62875/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:53.928613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f2061-787c-4935-b8a7-4ca3af12f545 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:53.937234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b82028-130e-4f43-8631-a473c40c2d0d None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:53.937526 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b82028-130e-4f43-8631-a473c40c2d0d None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:53.937806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b82028-130e-4f43-8631-a473c40c2d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:53.964546 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-c3b82028-130e-4f43-8631-a473c40c2d0d None None] MFA Rules not processed for user `504e7605c1f741609bae7c70d4d62875`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:53.965759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-c3b82028-130e-4f43-8631-a473c40c2d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:53.985657 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916fe1a-62d1-4edc-a62f-07233148b0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:53.986593 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916fe1a-62d1-4edc-a62f-07233148b0dd admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:53.986677 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916fe1a-62d1-4edc-a62f-07233148b0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:53.986910 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916fe1a-62d1-4edc-a62f-07233148b0dd admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:53.987650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916fe1a-62d1-4edc-a62f-07233148b0dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:53.990595 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916fe1a-62d1-4edc-a62f-07233148b0dd admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:54.060124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 993/1977] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:54.065267 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b82028-130e-4f43-8631-a473c40c2d0d None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:54.076648 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b82028-130e-4f43-8631-a473c40c2d0d None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:54.081643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5334d60-14a8-435c-a4a4-d6433e0ac835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:54.082349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5334d60-14a8-435c-a4a4-d6433e0ac835 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:54.082763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5334d60-14a8-435c-a4a4-d6433e0ac835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:54.083041 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5334d60-14a8-435c-a4a4-d6433e0ac835 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:54.083746 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5334d60-14a8-435c-a4a4-d6433e0ac835 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:54.085886 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5334d60-14a8-435c-a4a4-d6433e0ac835 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:54.102076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 985/1978] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:53 2022] POST /identity/v3/users => generated 287 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:17:54.108064 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916fe1a-62d1-4edc-a62f-07233148b0dd admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:54.119708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916fe1a-62d1-4edc-a62f-07233148b0dd admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:54.124422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41aa49-67cf-4c70-b049-3272d0fb5878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:54.125094 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41aa49-67cf-4c70-b049-3272d0fb5878 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:54.126125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41aa49-67cf-4c70-b049-3272d0fb5878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:54.126415 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41aa49-67cf-4c70-b049-3272d0fb5878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:54.127192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41aa49-67cf-4c70-b049-3272d0fb5878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:54.128841 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41aa49-67cf-4c70-b049-3272d0fb5878 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:54.139149 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 986/1979] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:17:54.143846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41aa49-67cf-4c70-b049-3272d0fb5878 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:54.153934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41aa49-67cf-4c70-b049-3272d0fb5878 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:54.159743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b1061-7f33-4751-b897-60b1366c0a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:54.160403 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b1061-7f33-4751-b897-60b1366c0a50 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:54.160646 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b1061-7f33-4751-b897-60b1366c0a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:54.160927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b1061-7f33-4751-b897-60b1366c0a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:54.162055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076b1061-7f33-4751-b897-60b1366c0a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:54.163298 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076b1061-7f33-4751-b897-60b1366c0a50 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:54.180925 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 987/1980] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:17:54.180925 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b1061-7f33-4751-b897-60b1366c0a50 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:54.190573 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b1061-7f33-4751-b897-60b1366c0a50 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:54.195696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765aedd-9613-4e3a-a1b9-80b3f9674bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:54.196619 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765aedd-9613-4e3a-a1b9-80b3f9674bb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:54.196906 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765aedd-9613-4e3a-a1b9-80b3f9674bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:54.197354 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765aedd-9613-4e3a-a1b9-80b3f9674bb9 admin admin] PATH_INFO: `/v3/projects/ab79a28775e4436cb0536db8a1f42bc3/users/e909e37cee73490d8f72c92b93e52f87/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:54.212693 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6765aedd-9613-4e3a-a1b9-80b3f9674bb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab79a28775e4436cb0536db8a1f42bc3, user_id=e909e37cee73490d8f72c92b93e52f87, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:54.215490 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6765aedd-9613-4e3a-a1b9-80b3f9674bb9 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:54.239736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 994/1981] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:54 2022] POST /identity/v3/projects => generated 387 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:54.242945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5334d60-14a8-435c-a4a4-d6433e0ac835 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:54.306124 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 988/1982] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:54 2022] PUT /identity/v3/projects/ab79a28775e4436cb0536db8a1f42bc3/users/e909e37cee73490d8f72c92b93e52f87/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:54.312003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5334d60-14a8-435c-a4a4-d6433e0ac835 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:54.318792 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9b2b8-bcbf-439d-b970-3cab60f2bec6 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:54.319102 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9b2b8-bcbf-439d-b970-3cab60f2bec6 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:54.319386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9b2b8-bcbf-439d-b970-3cab60f2bec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:54.345968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-1be9b2b8-bcbf-439d-b970-3cab60f2bec6 None None] MFA Rules not processed for user `e909e37cee73490d8f72c92b93e52f87`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:54.347159 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1be9b2b8-bcbf-439d-b970-3cab60f2bec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:54.368863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c83e7-e5ce-4e07-af71-deb7b3e1c0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:54.369539 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c83e7-e5ce-4e07-af71-deb7b3e1c0cc admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:54.369823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c83e7-e5ce-4e07-af71-deb7b3e1c0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:54.370140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c83e7-e5ce-4e07-af71-deb7b3e1c0cc admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:54.370844 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c83e7-e5ce-4e07-af71-deb7b3e1c0cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:54.372777 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c83e7-e5ce-4e07-af71-deb7b3e1c0cc admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:54.411735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 989/1983] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:54.423828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9b2b8-bcbf-439d-b970-3cab60f2bec6 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:54.425193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1be9b2b8-bcbf-439d-b970-3cab60f2bec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:54.480391 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9b2b8-bcbf-439d-b970-3cab60f2bec6 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:54.484117 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764ca31-ce51-49a2-b216-958b08b07d6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:54.484766 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764ca31-ce51-49a2-b216-958b08b07d6d service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:54.485695 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764ca31-ce51-49a2-b216-958b08b07d6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:54.485941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764ca31-ce51-49a2-b216-958b08b07d6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:54.491120 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 995/1984] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:54 2022] POST /identity/v3/users => generated 300 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:54.492227 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c83e7-e5ce-4e07-af71-deb7b3e1c0cc admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:54.494737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764ca31-ce51-49a2-b216-958b08b07d6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:54.496183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764ca31-ce51-49a2-b216-958b08b07d6d service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:54.502636 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c83e7-e5ce-4e07-af71-deb7b3e1c0cc admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:54.506025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 990/1985] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:17:54.508324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b7072-0e55-4041-be0a-0886ccb5e820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:54.509029 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b7072-0e55-4041-be0a-0886ccb5e820 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:54.509364 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b7072-0e55-4041-be0a-0886ccb5e820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:54.509634 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b7072-0e55-4041-be0a-0886ccb5e820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:54.510403 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b7072-0e55-4041-be0a-0886ccb5e820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:54.516929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b7072-0e55-4041-be0a-0886ccb5e820 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:54.526471 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 996/1986] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:17:54.530334 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764ca31-ce51-49a2-b216-958b08b07d6d service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:54.536342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b7072-0e55-4041-be0a-0886ccb5e820 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:54.541286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764ca31-ce51-49a2-b216-958b08b07d6d service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:54.546592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b7072-0e55-4041-be0a-0886ccb5e820 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:54.547691 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e08096-d6df-47d0-8c7a-206ad134b459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:54.548575 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e08096-d6df-47d0-8c7a-206ad134b459 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:54.548861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e08096-d6df-47d0-8c7a-206ad134b459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:54.549139 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e08096-d6df-47d0-8c7a-206ad134b459 admin admin] PATH_INFO: `/v3/projects/6e2bb5f6516f4290a0a44e264f5f4b6d/users/def1d7428dfc4018abe005dafb75a60f/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:54.551372 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827367a2-e128-48b2-a64e-5b786f9558ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:54.552155 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827367a2-e128-48b2-a64e-5b786f9558ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:54.552444 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827367a2-e128-48b2-a64e-5b786f9558ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:54.552725 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827367a2-e128-48b2-a64e-5b786f9558ca admin admin] PATH_INFO: `/v3/users/e909e37cee73490d8f72c92b93e52f87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:54.554060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827367a2-e128-48b2-a64e-5b786f9558ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=e909e37cee73490d8f72c92b93e52f87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:54.556913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827367a2-e128-48b2-a64e-5b786f9558ca admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:54.566504 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e08096-d6df-47d0-8c7a-206ad134b459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e2bb5f6516f4290a0a44e264f5f4b6d, user_id=def1d7428dfc4018abe005dafb75a60f, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:54.569146 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e08096-d6df-47d0-8c7a-206ad134b459 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:54.730765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 991/1987] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:54 2022] PUT /identity/v3/projects/6e2bb5f6516f4290a0a44e264f5f4b6d/users/def1d7428dfc4018abe005dafb75a60f/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:54.734823 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e08096-d6df-47d0-8c7a-206ad134b459 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:54.795592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e08096-d6df-47d0-8c7a-206ad134b459 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:54.800778 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250a865-4bc0-442b-b3bc-b2e34121dd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:54.801434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250a865-4bc0-442b-b3bc-b2e34121dd99 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:54.801714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250a865-4bc0-442b-b3bc-b2e34121dd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:54.801983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250a865-4bc0-442b-b3bc-b2e34121dd99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:54.802736 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250a865-4bc0-442b-b3bc-b2e34121dd99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:54.804307 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250a865-4bc0-442b-b3bc-b2e34121dd99 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:54.815137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 992/1988] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:17:54.819239 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250a865-4bc0-442b-b3bc-b2e34121dd99 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:54.828941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250a865-4bc0-442b-b3bc-b2e34121dd99 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:54.833628 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb172e67-3e78-420a-8183-d1ad6c50c2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:54.834514 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb172e67-3e78-420a-8183-d1ad6c50c2b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:54.834802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb172e67-3e78-420a-8183-d1ad6c50c2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:54.835079 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb172e67-3e78-420a-8183-d1ad6c50c2b6 admin admin] PATH_INFO: `/v3/projects/6e2bb5f6516f4290a0a44e264f5f4b6d/users/def1d7428dfc4018abe005dafb75a60f/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:54.837059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb172e67-3e78-420a-8183-d1ad6c50c2b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e2bb5f6516f4290a0a44e264f5f4b6d, user_id=def1d7428dfc4018abe005dafb75a60f, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:54.839797 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb172e67-3e78-420a-8183-d1ad6c50c2b6 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:54.850147 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 993/1989] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:54 2022] PUT /identity/v3/projects/6e2bb5f6516f4290a0a44e264f5f4b6d/users/def1d7428dfc4018abe005dafb75a60f/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:54.854298 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-827367a2-e128-48b2-a64e-5b786f9558ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e909e37cee73490d8f72c92b93e52f87'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:54.854943 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-827367a2-e128-48b2-a64e-5b786f9558ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e909e37cee73490d8f72c92b93e52f87'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:54.855554 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-827367a2-e128-48b2-a64e-5b786f9558ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e909e37cee73490d8f72c92b93e52f87'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:54.861268 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec735700-7240-496f-a7ff-fbe6392c57b8 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:54.861932 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec735700-7240-496f-a7ff-fbe6392c57b8 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:54.862275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec735700-7240-496f-a7ff-fbe6392c57b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:54.868253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-827367a2-e128-48b2-a64e-5b786f9558ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e909e37cee73490d8f72c92b93e52f87'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:54.875025 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-827367a2-e128-48b2-a64e-5b786f9558ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e909e37cee73490d8f72c92b93e52f87'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:54.888717 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-ec735700-7240-496f-a7ff-fbe6392c57b8 None None] MFA Rules not processed for user `def1d7428dfc4018abe005dafb75a60f`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:54.889965 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-ec735700-7240-496f-a7ff-fbe6392c57b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:54.956433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 994/1990] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:54.961840 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 997/1991] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:54 2022] DELETE /identity/v3/users/e909e37cee73490d8f72c92b93e52f87 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:54.969982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec735700-7240-496f-a7ff-fbe6392c57b8 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:54.970577 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8963b2c-8476-407d-8a43-93e6631d570e None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:54.970970 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8963b2c-8476-407d-8a43-93e6631d570e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:54.971246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8963b2c-8476-407d-8a43-93e6631d570e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:54.971504 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-ec735700-7240-496f-a7ff-fbe6392c57b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:54.988328 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-c8963b2c-8476-407d-8a43-93e6631d570e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:54.989555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-c8963b2c-8476-407d-8a43-93e6631d570e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:55.026271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec735700-7240-496f-a7ff-fbe6392c57b8 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:55.049897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 998/1992] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:55.062348 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8963b2c-8476-407d-8a43-93e6631d570e None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:55.064292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-c8963b2c-8476-407d-8a43-93e6631d570e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:55.079967 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4f1c1-ad3c-4527-806e-c057f518df99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:55.080798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4f1c1-ad3c-4527-806e-c057f518df99 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:55.081126 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4f1c1-ad3c-4527-806e-c057f518df99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:55.081445 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4f1c1-ad3c-4527-806e-c057f518df99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:55.124307 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8963b2c-8476-407d-8a43-93e6631d570e None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:55.128238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208e360-4baf-47b6-841b-96fcbf320f9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:55.128894 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208e360-4baf-47b6-841b-96fcbf320f9e service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:55.129180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208e360-4baf-47b6-841b-96fcbf320f9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:55.129459 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208e360-4baf-47b6-841b-96fcbf320f9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:55.138894 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208e360-4baf-47b6-841b-96fcbf320f9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:55.144799 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208e360-4baf-47b6-841b-96fcbf320f9e service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:55.144907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f4f1c1-ad3c-4527-806e-c057f518df99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:55.144907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f4f1c1-ad3c-4527-806e-c057f518df99 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:55.150850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 999/1993] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:17:55.154106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 995/1994] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:17:55.226385 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208e360-4baf-47b6-841b-96fcbf320f9e service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:55.226385 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3208e360-4baf-47b6-841b-96fcbf320f9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:55.239016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208e360-4baf-47b6-841b-96fcbf320f9e service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:55.243747 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec612ec-c4b5-4559-ae71-073134450f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:55.244639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec612ec-c4b5-4559-ae71-073134450f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:55.244923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec612ec-c4b5-4559-ae71-073134450f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:55.245197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec612ec-c4b5-4559-ae71-073134450f60 admin admin] PATH_INFO: `/v3/users/504e7605c1f741609bae7c70d4d62875` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:55.246536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec612ec-c4b5-4559-ae71-073134450f60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=504e7605c1f741609bae7c70d4d62875)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:55.248483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec612ec-c4b5-4559-ae71-073134450f60 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:55.444337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1ec612ec-c4b5-4559-ae71-073134450f60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '504e7605c1f741609bae7c70d4d62875'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:55.445100 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1ec612ec-c4b5-4559-ae71-073134450f60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '504e7605c1f741609bae7c70d4d62875'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:55.446370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1ec612ec-c4b5-4559-ae71-073134450f60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '504e7605c1f741609bae7c70d4d62875'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:55.458749 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1ec612ec-c4b5-4559-ae71-073134450f60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '504e7605c1f741609bae7c70d4d62875'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:55.465757 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1ec612ec-c4b5-4559-ae71-073134450f60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '504e7605c1f741609bae7c70d4d62875'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:55.561447 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1000/1995] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:55 2022] DELETE /identity/v3/users/504e7605c1f741609bae7c70d4d62875 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:55.566481 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b2b3d-669d-4f27-8c87-7c4096632978 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:55.566776 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b2b3d-669d-4f27-8c87-7c4096632978 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:55.567061 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b2b3d-669d-4f27-8c87-7c4096632978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:55.584007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-459b2b3d-669d-4f27-8c87-7c4096632978 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:55.585214 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-459b2b3d-669d-4f27-8c87-7c4096632978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:55.592137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec612ec-c4b5-4559-ae71-073134450f60 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:55.594188 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-1ec612ec-c4b5-4559-ae71-073134450f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:55.654549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 996/1996] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:55.664660 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b2b3d-669d-4f27-8c87-7c4096632978 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:55.666059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-459b2b3d-669d-4f27-8c87-7c4096632978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:55.693511 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec612ec-c4b5-4559-ae71-073134450f60 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:55.698499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6c8b6-3f66-495c-a69f-7b9d40128483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:55.699409 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6c8b6-3f66-495c-a69f-7b9d40128483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:55.699697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6c8b6-3f66-495c-a69f-7b9d40128483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:55.700024 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6c8b6-3f66-495c-a69f-7b9d40128483 admin admin] PATH_INFO: `/v3/projects/ab79a28775e4436cb0536db8a1f42bc3` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:55.701249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6c8b6-3f66-495c-a69f-7b9d40128483 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab79a28775e4436cb0536db8a1f42bc3)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:55.703202 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6c8b6-3f66-495c-a69f-7b9d40128483 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:55.722628 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b2b3d-669d-4f27-8c87-7c4096632978 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:55.726295 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405c2f3-8f39-4dad-bf32-037f605c9037 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:55.726944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405c2f3-8f39-4dad-bf32-037f605c9037 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:55.727231 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405c2f3-8f39-4dad-bf32-037f605c9037 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:55.727622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405c2f3-8f39-4dad-bf32-037f605c9037 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:55.737267 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405c2f3-8f39-4dad-bf32-037f605c9037 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:55.738749 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405c2f3-8f39-4dad-bf32-037f605c9037 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:55.747966 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 997/1997] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:17:55.828844 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a7b6c8b6-3f66-495c-a69f-7b9d40128483 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab79a28775e4436cb0536db8a1f42bc3', 'request_id': 'req-a7b6c8b6-3f66-495c-a69f-7b9d40128483'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:55.881362 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a7b6c8b6-3f66-495c-a69f-7b9d40128483 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab79a28775e4436cb0536db8a1f42bc3', 'request_id': 'req-a7b6c8b6-3f66-495c-a69f-7b9d40128483'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:55.887155 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a7b6c8b6-3f66-495c-a69f-7b9d40128483 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab79a28775e4436cb0536db8a1f42bc3', 'request_id': 'req-a7b6c8b6-3f66-495c-a69f-7b9d40128483'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:55.898250 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a7b6c8b6-3f66-495c-a69f-7b9d40128483 admin admin] The token cache is being invalidate because project ab79a28775e4436cb0536db8a1f42bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:55.898539 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a7b6c8b6-3f66-495c-a69f-7b9d40128483 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:55.899179 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a7b6c8b6-3f66-495c-a69f-7b9d40128483 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:55.900768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1001/1998] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:55 2022] DELETE /identity/v3/projects/ab79a28775e4436cb0536db8a1f42bc3 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:55.905875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685b58b-33cc-4689-9bc1-87accb24fe3b None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:55.906372 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685b58b-33cc-4689-9bc1-87accb24fe3b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:55.906592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685b58b-33cc-4689-9bc1-87accb24fe3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:55.926337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-b685b58b-33cc-4689-9bc1-87accb24fe3b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:55.927751 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b685b58b-33cc-4689-9bc1-87accb24fe3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:55.991638 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 998/1999] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:55.999293 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c31bb-392f-46f1-a4b6-4fd0d5f09aaa None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:55.999584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c31bb-392f-46f1-a4b6-4fd0d5f09aaa None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:55.999891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c31bb-392f-46f1-a4b6-4fd0d5f09aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:56.021477 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-d90c31bb-392f-46f1-a4b6-4fd0d5f09aaa None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:56.029975 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-d90c31bb-392f-46f1-a4b6-4fd0d5f09aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:56.036996 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1002/2000] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:56.041481 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685b58b-33cc-4689-9bc1-87accb24fe3b None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:56.052095 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685b58b-33cc-4689-9bc1-87accb24fe3b None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:56.057746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9309f26-7d47-47e2-acc4-6b93b6dc20c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:56.058448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9309f26-7d47-47e2-acc4-6b93b6dc20c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:56.058852 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9309f26-7d47-47e2-acc4-6b93b6dc20c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:56.059139 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9309f26-7d47-47e2-acc4-6b93b6dc20c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:56.059884 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9309f26-7d47-47e2-acc4-6b93b6dc20c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:17:56.060196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9309f26-7d47-47e2-acc4-6b93b6dc20c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:56.062798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9309f26-7d47-47e2-acc4-6b93b6dc20c0 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:56.084000 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 999/2001] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:17:56.088231 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c31bb-392f-46f1-a4b6-4fd0d5f09aaa None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:56.099415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c31bb-392f-46f1-a4b6-4fd0d5f09aaa None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:56.104665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a4ec9-9589-4b9a-b52f-3ad0933d97ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:56.105335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a4ec9-9589-4b9a-b52f-3ad0933d97ff admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:56.105623 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a4ec9-9589-4b9a-b52f-3ad0933d97ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:56.105901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a4ec9-9589-4b9a-b52f-3ad0933d97ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:56.106842 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a4ec9-9589-4b9a-b52f-3ad0933d97ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:56.108962 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a4ec9-9589-4b9a-b52f-3ad0933d97ff admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:56.214674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1003/2002] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:56 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:56.214993 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9309f26-7d47-47e2-acc4-6b93b6dc20c0 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:56.216944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c9309f26-7d47-47e2-acc4-6b93b6dc20c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:56.218656 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a4ec9-9589-4b9a-b52f-3ad0933d97ff admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:56.241129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9309f26-7d47-47e2-acc4-6b93b6dc20c0 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:56.288524 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a4ec9-9589-4b9a-b52f-3ad0933d97ff admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:56.293119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c05dfe-2f20-4e14-aa51-6b7d1f1a1cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:56.293781 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c05dfe-2f20-4e14-aa51-6b7d1f1a1cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:56.294068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c05dfe-2f20-4e14-aa51-6b7d1f1a1cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:56.294372 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c05dfe-2f20-4e14-aa51-6b7d1f1a1cb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:56.295066 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c05dfe-2f20-4e14-aa51-6b7d1f1a1cb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:56.296761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34f7f7-9181-40d2-8561-497f413d43b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:56.297094 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c05dfe-2f20-4e14-aa51-6b7d1f1a1cb5 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:56.297740 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34f7f7-9181-40d2-8561-497f413d43b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:56.298568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34f7f7-9181-40d2-8561-497f413d43b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:56.298847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34f7f7-9181-40d2-8561-497f413d43b7 admin admin] PATH_INFO: `/v3/projects/64e4263d76ee40548fec18021cee5b10` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:56.300051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34f7f7-9181-40d2-8561-497f413d43b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e4263d76ee40548fec18021cee5b10)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:56.301936 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34f7f7-9181-40d2-8561-497f413d43b7 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:56.405368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1004/2003] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:56 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:17:56.409793 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c05dfe-2f20-4e14-aa51-6b7d1f1a1cb5 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:56.420039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c05dfe-2f20-4e14-aa51-6b7d1f1a1cb5 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:56.433826 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4d34f7f7-9181-40d2-8561-497f413d43b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e4263d76ee40548fec18021cee5b10', 'request_id': 'req-4d34f7f7-9181-40d2-8561-497f413d43b7'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:56.478043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d87ccc-3479-4518-9437-6152810cf81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:56.478806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d87ccc-3479-4518-9437-6152810cf81f admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:56.479092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d87ccc-3479-4518-9437-6152810cf81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:56.479366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d87ccc-3479-4518-9437-6152810cf81f admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:56.480102 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d87ccc-3479-4518-9437-6152810cf81f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:56.481688 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d87ccc-3479-4518-9437-6152810cf81f admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:56.491699 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1005/2004] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:17:56.495594 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d87ccc-3479-4518-9437-6152810cf81f admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:56.505699 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d87ccc-3479-4518-9437-6152810cf81f admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:56.510465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef6879-bdbb-4f4a-81d8-9c529a3b38fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:56.511011 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4d34f7f7-9181-40d2-8561-497f413d43b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e4263d76ee40548fec18021cee5b10', 'request_id': 'req-4d34f7f7-9181-40d2-8561-497f413d43b7'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:56.511124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef6879-bdbb-4f4a-81d8-9c529a3b38fe admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:56.511401 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef6879-bdbb-4f4a-81d8-9c529a3b38fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:56.511675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef6879-bdbb-4f4a-81d8-9c529a3b38fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:56.512409 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef6879-bdbb-4f4a-81d8-9c529a3b38fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:56.513997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef6879-bdbb-4f4a-81d8-9c529a3b38fe admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:56.518828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4d34f7f7-9181-40d2-8561-497f413d43b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e4263d76ee40548fec18021cee5b10', 'request_id': 'req-4d34f7f7-9181-40d2-8561-497f413d43b7'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:56.528013 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1006/2005] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:17:56.528810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef6879-bdbb-4f4a-81d8-9c529a3b38fe admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:56.531083 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4d34f7f7-9181-40d2-8561-497f413d43b7 admin admin] The token cache is being invalidate because project 64e4263d76ee40548fec18021cee5b10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:56.531381 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4d34f7f7-9181-40d2-8561-497f413d43b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:56.532723 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4d34f7f7-9181-40d2-8561-497f413d43b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:56.534668 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1000/2006] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:56 2022] DELETE /identity/v3/projects/64e4263d76ee40548fec18021cee5b10 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:56.538529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34f7f7-9181-40d2-8561-497f413d43b7 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:56.539656 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef6879-bdbb-4f4a-81d8-9c529a3b38fe admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:56.539994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-4d34f7f7-9181-40d2-8561-497f413d43b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:56.544435 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634982dc-4170-43dc-bab4-8a76e752877b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:56.545313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634982dc-4170-43dc-bab4-8a76e752877b admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:56.545596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634982dc-4170-43dc-bab4-8a76e752877b admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:56.545869 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634982dc-4170-43dc-bab4-8a76e752877b admin admin] PATH_INFO: `/v3/projects/b135986dfe1f499a9655ac3faee4bbbf/users/6f5b649235d646fea59e7e61e7e6d8a9/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:56.553781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34f7f7-9181-40d2-8561-497f413d43b7 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:56.559241 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd4cb2-4b3e-466a-93d5-772f0b23257a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:56.560115 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd4cb2-4b3e-466a-93d5-772f0b23257a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:56.560421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd4cb2-4b3e-466a-93d5-772f0b23257a admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:56.560695 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd4cb2-4b3e-466a-93d5-772f0b23257a admin admin] PATH_INFO: `/v3/users/def1d7428dfc4018abe005dafb75a60f` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:56.561706 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634982dc-4170-43dc-bab4-8a76e752877b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b135986dfe1f499a9655ac3faee4bbbf, user_id=6f5b649235d646fea59e7e61e7e6d8a9, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:56.561906 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd4cb2-4b3e-466a-93d5-772f0b23257a admin admin] RBAC: Authorizing `identity:delete_user(user_id=def1d7428dfc4018abe005dafb75a60f)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:56.564077 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd4cb2-4b3e-466a-93d5-772f0b23257a admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:56.564548 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634982dc-4170-43dc-bab4-8a76e752877b admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:56.636449 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1007/2007] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:56 2022] PUT /identity/v3/projects/b135986dfe1f499a9655ac3faee4bbbf/users/6f5b649235d646fea59e7e61e7e6d8a9/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:17:56.641071 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5c7c8-3c3f-4ce4-80c1-71e3d825668c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:56.645599 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5c7c8-3c3f-4ce4-80c1-71e3d825668c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:56.645599 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5c7c8-3c3f-4ce4-80c1-71e3d825668c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:56.659172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-cde5c7c8-3c3f-4ce4-80c1-71e3d825668c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:56.660365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-cde5c7c8-3c3f-4ce4-80c1-71e3d825668c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:56.731104 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1008/2008] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:17:56.740836 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5c7c8-3c3f-4ce4-80c1-71e3d825668c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:56.742203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-cde5c7c8-3c3f-4ce4-80c1-71e3d825668c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:56.753652 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-59fd4cb2-4b3e-466a-93d5-772f0b23257a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'def1d7428dfc4018abe005dafb75a60f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:56.754395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-59fd4cb2-4b3e-466a-93d5-772f0b23257a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'def1d7428dfc4018abe005dafb75a60f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:56.769167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-59fd4cb2-4b3e-466a-93d5-772f0b23257a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'def1d7428dfc4018abe005dafb75a60f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:56.769846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-59fd4cb2-4b3e-466a-93d5-772f0b23257a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'def1d7428dfc4018abe005dafb75a60f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:56.776654 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-59fd4cb2-4b3e-466a-93d5-772f0b23257a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'def1d7428dfc4018abe005dafb75a60f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:56.798343 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5c7c8-3c3f-4ce4-80c1-71e3d825668c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:56.802061 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366da21-6a4a-4de2-9acc-06843e098ba5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:56.802727 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366da21-6a4a-4de2-9acc-06843e098ba5 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:56.803016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366da21-6a4a-4de2-9acc-06843e098ba5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:56.803300 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366da21-6a4a-4de2-9acc-06843e098ba5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:56.805109 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-6366da21-6a4a-4de2-9acc-06843e098ba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:56.817616 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366da21-6a4a-4de2-9acc-06843e098ba5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:56.819134 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366da21-6a4a-4de2-9acc-06843e098ba5 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:56.830787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1009/2009] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:17:56.831031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1001/2010] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:17:56 2022] DELETE /identity/v3/users/def1d7428dfc4018abe005dafb75a60f => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:57.358539 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366da21-6a4a-4de2-9acc-06843e098ba5 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:57.359970 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-6366da21-6a4a-4de2-9acc-06843e098ba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:57.372481 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366da21-6a4a-4de2-9acc-06843e098ba5 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:57.459520 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8fbcad-ca84-41a9-8e69-d23449f6a31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:57.460482 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fbcad-ca84-41a9-8e69-d23449f6a31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:57.460837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fbcad-ca84-41a9-8e69-d23449f6a31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:57.461182 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fbcad-ca84-41a9-8e69-d23449f6a31f admin admin] PATH_INFO: `/v3/projects/6e2bb5f6516f4290a0a44e264f5f4b6d` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:57.463894 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8fbcad-ca84-41a9-8e69-d23449f6a31f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e2bb5f6516f4290a0a44e264f5f4b6d)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:57.465852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8fbcad-ca84-41a9-8e69-d23449f6a31f admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:57.588026 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ae8fbcad-ca84-41a9-8e69-d23449f6a31f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e2bb5f6516f4290a0a44e264f5f4b6d', 'request_id': 'req-ae8fbcad-ca84-41a9-8e69-d23449f6a31f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:57.652850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ae8fbcad-ca84-41a9-8e69-d23449f6a31f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e2bb5f6516f4290a0a44e264f5f4b6d', 'request_id': 'req-ae8fbcad-ca84-41a9-8e69-d23449f6a31f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:57.659935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ae8fbcad-ca84-41a9-8e69-d23449f6a31f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e2bb5f6516f4290a0a44e264f5f4b6d', 'request_id': 'req-ae8fbcad-ca84-41a9-8e69-d23449f6a31f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:57.690349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ae8fbcad-ca84-41a9-8e69-d23449f6a31f admin admin] The token cache is being invalidate because project 6e2bb5f6516f4290a0a44e264f5f4b6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:17:57.690349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ae8fbcad-ca84-41a9-8e69-d23449f6a31f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:57.690349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ae8fbcad-ca84-41a9-8e69-d23449f6a31f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:17:57.692714 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1010/2011] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:17:57 2022] DELETE /identity/v3/projects/6e2bb5f6516f4290a0a44e264f5f4b6d => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:57.698608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba68d99-7d0e-458f-8bdc-f63e4765da43 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:57.699105 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba68d99-7d0e-458f-8bdc-f63e4765da43 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:57.699105 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba68d99-7d0e-458f-8bdc-f63e4765da43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:57.718283 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-fba68d99-7d0e-458f-8bdc-f63e4765da43 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:57.723612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-fba68d99-7d0e-458f-8bdc-f63e4765da43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:57.784889 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1002/2012] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:57.793630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6112045-2e82-42ac-89a1-2d3ddf8179ec None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:57.793926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6112045-2e82-42ac-89a1-2d3ddf8179ec None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:57.794234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6112045-2e82-42ac-89a1-2d3ddf8179ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:57.814746 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-a6112045-2e82-42ac-89a1-2d3ddf8179ec None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:57.815968 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a6112045-2e82-42ac-89a1-2d3ddf8179ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:57.824858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1011/2013] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:57.829102 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba68d99-7d0e-458f-8bdc-f63e4765da43 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:57.838864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba68d99-7d0e-458f-8bdc-f63e4765da43 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:57.843531 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba34570-798a-46d2-a2b6-7f693c0d7615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:57.844197 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba34570-798a-46d2-a2b6-7f693c0d7615 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:57.844485 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba34570-798a-46d2-a2b6-7f693c0d7615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:57.844791 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba34570-798a-46d2-a2b6-7f693c0d7615 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:57.845527 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba34570-798a-46d2-a2b6-7f693c0d7615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:17:57.845835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba34570-798a-46d2-a2b6-7f693c0d7615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:57.847481 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba34570-798a-46d2-a2b6-7f693c0d7615 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:57.859311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1003/2014] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:17:57.864243 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6112045-2e82-42ac-89a1-2d3ddf8179ec None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:57.874463 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6112045-2e82-42ac-89a1-2d3ddf8179ec None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:57.882986 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461a7f3-8932-4638-a886-a6e0ae757225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:57.883719 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461a7f3-8932-4638-a886-a6e0ae757225 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:57.884000 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461a7f3-8932-4638-a886-a6e0ae757225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:57.884273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461a7f3-8932-4638-a886-a6e0ae757225 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:57.884993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461a7f3-8932-4638-a886-a6e0ae757225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:57.886979 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461a7f3-8932-4638-a886-a6e0ae757225 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:57.972467 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1012/2015] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:17:57 2022] POST /identity/v3/projects => generated 363 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:57.976883 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba34570-798a-46d2-a2b6-7f693c0d7615 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:57.987414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba34570-798a-46d2-a2b6-7f693c0d7615 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:58.046205 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0904e61-456f-497d-a957-a1a24642ef65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:58.046886 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0904e61-456f-497d-a957-a1a24642ef65 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:58.047167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0904e61-456f-497d-a957-a1a24642ef65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:58.047717 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0904e61-456f-497d-a957-a1a24642ef65 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:58.049006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0904e61-456f-497d-a957-a1a24642ef65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:58.051626 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0904e61-456f-497d-a957-a1a24642ef65 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:58.155590 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1004/2016] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:17:57 2022] POST /identity/v3/users => generated 282 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:17:58.160778 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461a7f3-8932-4638-a886-a6e0ae757225 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:58.171134 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461a7f3-8932-4638-a886-a6e0ae757225 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:58.176462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b221d-744a-4650-a36f-7276bd446831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:58.177126 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b221d-744a-4650-a36f-7276bd446831 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:58.177436 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b221d-744a-4650-a36f-7276bd446831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:58.177722 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b221d-744a-4650-a36f-7276bd446831 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:58.178496 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b221d-744a-4650-a36f-7276bd446831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:58.180094 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b221d-744a-4650-a36f-7276bd446831 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:58.190947 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1013/2017] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:58.195057 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0904e61-456f-497d-a957-a1a24642ef65 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:58.205090 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0904e61-456f-497d-a957-a1a24642ef65 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:58.209950 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcf360-80b6-48de-a1f5-4245c6460a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:58.210657 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcf360-80b6-48de-a1f5-4245c6460a10 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:58.211032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcf360-80b6-48de-a1f5-4245c6460a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:58.211221 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcf360-80b6-48de-a1f5-4245c6460a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:58.211954 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fcf360-80b6-48de-a1f5-4245c6460a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:58.213568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fcf360-80b6-48de-a1f5-4245c6460a10 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:58.225643 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1005/2018] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:17:58.229658 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b221d-744a-4650-a36f-7276bd446831 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:58.240638 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b221d-744a-4650-a36f-7276bd446831 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:58.245453 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a9db2-3e53-4d5a-ab7c-a4f039305570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:58.246405 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a9db2-3e53-4d5a-ab7c-a4f039305570 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:58.246838 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a9db2-3e53-4d5a-ab7c-a4f039305570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:58.251741 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a9db2-3e53-4d5a-ab7c-a4f039305570 admin admin] PATH_INFO: `/v3/projects/c3d8d2fe430e47a3bdca084e051efc72/users/2ffb8c46abaa474483a2c4e3926bd1fa/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:58.262593 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a9db2-3e53-4d5a-ab7c-a4f039305570 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d8d2fe430e47a3bdca084e051efc72, user_id=2ffb8c46abaa474483a2c4e3926bd1fa, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:58.265370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a9db2-3e53-4d5a-ab7c-a4f039305570 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:58.334006 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1014/2019] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:17:58 2022] PUT /identity/v3/projects/c3d8d2fe430e47a3bdca084e051efc72/users/2ffb8c46abaa474483a2c4e3926bd1fa/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:17:58.345329 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d563c3-62b9-4a1f-8d60-ad7887bc114d None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:58.345703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d563c3-62b9-4a1f-8d60-ad7887bc114d None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:58.346619 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d563c3-62b9-4a1f-8d60-ad7887bc114d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:58.375233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-13d563c3-62b9-4a1f-8d60-ad7887bc114d None None] MFA Rules not processed for user `2ffb8c46abaa474483a2c4e3926bd1fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:17:58.376781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-13d563c3-62b9-4a1f-8d60-ad7887bc114d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:58.459557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1006/2020] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:17:58 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:17:58.471252 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a9db2-3e53-4d5a-ab7c-a4f039305570 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:17:58.472947 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-004a9db2-3e53-4d5a-ab7c-a4f039305570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:17:58.528908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a9db2-3e53-4d5a-ab7c-a4f039305570 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:17:58.533376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1012b0-f6a9-422c-b905-17389b95ff21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:17:58.534026 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1012b0-f6a9-422c-b905-17389b95ff21 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:17:58.534370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1012b0-f6a9-422c-b905-17389b95ff21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:17:58.534678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1012b0-f6a9-422c-b905-17389b95ff21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:17:58.543540 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1012b0-f6a9-422c-b905-17389b95ff21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:17:58.545288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1012b0-f6a9-422c-b905-17389b95ff21 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:17:58.557910 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1015/2021] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:17:58 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:03.071312 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e14a01-29b3-41b8-8aee-f5c5196bf762 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:03.071831 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e14a01-29b3-41b8-8aee-f5c5196bf762 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:03.071909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e14a01-29b3-41b8-8aee-f5c5196bf762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:03.117573 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-83e14a01-29b3-41b8-8aee-f5c5196bf762 None None] MFA Rules not processed for user `6f5b649235d646fea59e7e61e7e6d8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:03.118845 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-83e14a01-29b3-41b8-8aee-f5c5196bf762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:03.210458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1007/2022] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:03.215734 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1012b0-f6a9-422c-b905-17389b95ff21 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:03.217305 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-2e1012b0-f6a9-422c-b905-17389b95ff21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:03.303550 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1012b0-f6a9-422c-b905-17389b95ff21 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:03.309064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06f4f2-2bb6-4216-bad9-8c8f5413e3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:03.310011 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06f4f2-2bb6-4216-bad9-8c8f5413e3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:03.310524 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06f4f2-2bb6-4216-bad9-8c8f5413e3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:03.311010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06f4f2-2bb6-4216-bad9-8c8f5413e3e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:03.312461 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06f4f2-2bb6-4216-bad9-8c8f5413e3e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:03.316431 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06f4f2-2bb6-4216-bad9-8c8f5413e3e6 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:03.322488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e14a01-29b3-41b8-8aee-f5c5196bf762 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:03.324249 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-83e14a01-29b3-41b8-8aee-f5c5196bf762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:03.344755 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e14a01-29b3-41b8-8aee-f5c5196bf762 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:03.365341 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc15466-0248-4c03-acd3-431def4c3605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:03.366461 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc15466-0248-4c03-acd3-431def4c3605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:03.366990 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc15466-0248-4c03-acd3-431def4c3605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:03.368954 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc15466-0248-4c03-acd3-431def4c3605 admin admin] PATH_INFO: `/v3/users/c9bc1ceb41a6485094ece96346398ef0` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:03.371233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc15466-0248-4c03-acd3-431def4c3605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9bc1ceb41a6485094ece96346398ef0)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:03.372765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc15466-0248-4c03-acd3-431def4c3605 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:03.441441 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1016/2023] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:03 2022] POST /identity/v3/projects => generated 383 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:18:03.445591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06f4f2-2bb6-4216-bad9-8c8f5413e3e6 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:03.508842 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06f4f2-2bb6-4216-bad9-8c8f5413e3e6 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:03.513636 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de20114-99e4-46d2-968b-1ff11be155bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:03.514337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de20114-99e4-46d2-968b-1ff11be155bf admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:03.514619 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de20114-99e4-46d2-968b-1ff11be155bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:03.514905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de20114-99e4-46d2-968b-1ff11be155bf admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:03.515596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de20114-99e4-46d2-968b-1ff11be155bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:03.518147 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de20114-99e4-46d2-968b-1ff11be155bf admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:03.632007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1017/2024] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:03 2022] POST /identity/v3/users => generated 298 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:03.636097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de20114-99e4-46d2-968b-1ff11be155bf admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:03.644926 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0dc15466-0248-4c03-acd3-431def4c3605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9bc1ceb41a6485094ece96346398ef0'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:03.645589 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0dc15466-0248-4c03-acd3-431def4c3605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9bc1ceb41a6485094ece96346398ef0'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:03.647570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de20114-99e4-46d2-968b-1ff11be155bf admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:03.652740 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb4dcaf-4aff-4716-bc57-544b073920b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:03.653555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4dcaf-4aff-4716-bc57-544b073920b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:03.653898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4dcaf-4aff-4716-bc57-544b073920b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:03.654219 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4dcaf-4aff-4716-bc57-544b073920b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:03.654956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb4dcaf-4aff-4716-bc57-544b073920b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:03.656565 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb4dcaf-4aff-4716-bc57-544b073920b8 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:03.663251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0dc15466-0248-4c03-acd3-431def4c3605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9bc1ceb41a6485094ece96346398ef0'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:03.663251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0dc15466-0248-4c03-acd3-431def4c3605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9bc1ceb41a6485094ece96346398ef0'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:03.667415 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0dc15466-0248-4c03-acd3-431def4c3605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9bc1ceb41a6485094ece96346398ef0'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:03.667536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1018/2025] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:18:03.671368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb4dcaf-4aff-4716-bc57-544b073920b8 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:03.673141 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7eb4dcaf-4aff-4716-bc57-544b073920b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:03.686897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb4dcaf-4aff-4716-bc57-544b073920b8 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:03.692212 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444911d-fc7f-4d89-a8b6-4d5ef3c1a1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:03.693088 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444911d-fc7f-4d89-a8b6-4d5ef3c1a1ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:03.693374 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444911d-fc7f-4d89-a8b6-4d5ef3c1a1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:03.694264 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444911d-fc7f-4d89-a8b6-4d5ef3c1a1ee admin admin] PATH_INFO: `/v3/projects/4b5708fb61854562988f5bc2dee5b34b/users/72111ba468f744b2a50b100e15f6c7f8/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:03.708794 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444911d-fc7f-4d89-a8b6-4d5ef3c1a1ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b5708fb61854562988f5bc2dee5b34b, user_id=72111ba468f744b2a50b100e15f6c7f8, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:03.715243 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444911d-fc7f-4d89-a8b6-4d5ef3c1a1ee admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:03.799737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1008/2026] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:03 2022] DELETE /identity/v3/users/c9bc1ceb41a6485094ece96346398ef0 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:03.818994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1019/2027] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:03 2022] PUT /identity/v3/projects/4b5708fb61854562988f5bc2dee5b34b/users/72111ba468f744b2a50b100e15f6c7f8/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:03.823546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc15466-0248-4c03-acd3-431def4c3605 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:03.891622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc15466-0248-4c03-acd3-431def4c3605 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:03.952472 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bb444-ef2d-46ec-b956-5d39a23899b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:03.953259 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bb444-ef2d-46ec-b956-5d39a23899b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:03.953617 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bb444-ef2d-46ec-b956-5d39a23899b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:03.953963 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bb444-ef2d-46ec-b956-5d39a23899b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:03.954800 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889bb444-ef2d-46ec-b956-5d39a23899b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:03.963566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889bb444-ef2d-46ec-b956-5d39a23899b1 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:03.978647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1009/2028] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:03.982806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444911d-fc7f-4d89-a8b6-4d5ef3c1a1ee admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:03.992971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444911d-fc7f-4d89-a8b6-4d5ef3c1a1ee admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:03.997640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4529a6-b38d-41be-a538-dad8d85b09b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:03.998536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4529a6-b38d-41be-a538-dad8d85b09b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:03.998823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4529a6-b38d-41be-a538-dad8d85b09b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:03.999094 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4529a6-b38d-41be-a538-dad8d85b09b7 admin admin] PATH_INFO: `/v3/projects/4b5708fb61854562988f5bc2dee5b34b/users/72111ba468f744b2a50b100e15f6c7f8/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:04.001068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4529a6-b38d-41be-a538-dad8d85b09b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b5708fb61854562988f5bc2dee5b34b, user_id=72111ba468f744b2a50b100e15f6c7f8, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:04.003792 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4529a6-b38d-41be-a538-dad8d85b09b7 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:04.013805 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1020/2029] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:03 2022] PUT /identity/v3/projects/4b5708fb61854562988f5bc2dee5b34b/users/72111ba468f744b2a50b100e15f6c7f8/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:04.073307 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bb444-ef2d-46ec-b956-5d39a23899b1 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:04.075915 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-889bb444-ef2d-46ec-b956-5d39a23899b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:04.091782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bb444-ef2d-46ec-b956-5d39a23899b1 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:04.178909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d83d65-c5d8-4563-bf1a-de030cb9436e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:04.180424 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d83d65-c5d8-4563-bf1a-de030cb9436e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:04.181217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d83d65-c5d8-4563-bf1a-de030cb9436e admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:04.181503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d83d65-c5d8-4563-bf1a-de030cb9436e admin admin] PATH_INFO: `/v3/projects/19fcef97d9fc42889ca1435f33cef96b` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:04.183488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d83d65-c5d8-4563-bf1a-de030cb9436e admin admin] RBAC: Authorizing `identity:delete_project(project_id=19fcef97d9fc42889ca1435f33cef96b)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:04.186012 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d83d65-c5d8-4563-bf1a-de030cb9436e admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:04.330711 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-07d83d65-c5d8-4563-bf1a-de030cb9436e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19fcef97d9fc42889ca1435f33cef96b', 'request_id': 'req-07d83d65-c5d8-4563-bf1a-de030cb9436e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:04.431212 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-07d83d65-c5d8-4563-bf1a-de030cb9436e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19fcef97d9fc42889ca1435f33cef96b', 'request_id': 'req-07d83d65-c5d8-4563-bf1a-de030cb9436e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:04.440097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-07d83d65-c5d8-4563-bf1a-de030cb9436e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19fcef97d9fc42889ca1435f33cef96b', 'request_id': 'req-07d83d65-c5d8-4563-bf1a-de030cb9436e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:04.454323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-07d83d65-c5d8-4563-bf1a-de030cb9436e admin admin] The token cache is being invalidate because project 19fcef97d9fc42889ca1435f33cef96b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:04.454720 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-07d83d65-c5d8-4563-bf1a-de030cb9436e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:04.455629 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-07d83d65-c5d8-4563-bf1a-de030cb9436e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:04.458076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1010/2030] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:04 2022] DELETE /identity/v3/projects/19fcef97d9fc42889ca1435f33cef96b => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:04.462932 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9807a-65bb-4b54-acfc-e4fb12de384d None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:04.463422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9807a-65bb-4b54-acfc-e4fb12de384d None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:04.463521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9807a-65bb-4b54-acfc-e4fb12de384d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:04.497023 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-c6a9807a-65bb-4b54-acfc-e4fb12de384d None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:04.498758 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-c6a9807a-65bb-4b54-acfc-e4fb12de384d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:04.571625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1021/2031] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:04.579935 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85831132-07fe-4505-878c-fd0577a33a94 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:04.580931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85831132-07fe-4505-878c-fd0577a33a94 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:04.581314 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85831132-07fe-4505-878c-fd0577a33a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:04.605588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-85831132-07fe-4505-878c-fd0577a33a94 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:04.607155 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-85831132-07fe-4505-878c-fd0577a33a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:04.617006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1011/2032] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:04.621357 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9807a-65bb-4b54-acfc-e4fb12de384d None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:04.632882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9807a-65bb-4b54-acfc-e4fb12de384d None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:04.638962 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e2265-0017-40e1-a31d-8494adf445eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:04.639639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e2265-0017-40e1-a31d-8494adf445eb admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:04.639966 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e2265-0017-40e1-a31d-8494adf445eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:04.640244 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e2265-0017-40e1-a31d-8494adf445eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:04.640986 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e2265-0017-40e1-a31d-8494adf445eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:18:04.641729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e2265-0017-40e1-a31d-8494adf445eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:04.644308 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e2265-0017-40e1-a31d-8494adf445eb admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:04.657421 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1022/2033] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:18:04.670147 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85831132-07fe-4505-878c-fd0577a33a94 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:04.680522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85831132-07fe-4505-878c-fd0577a33a94 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:04.686990 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4ca50-ecd6-48b1-b477-95456254764f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:04.687677 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4ca50-ecd6-48b1-b477-95456254764f admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:04.687969 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4ca50-ecd6-48b1-b477-95456254764f admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:04.688249 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4ca50-ecd6-48b1-b477-95456254764f admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:04.689012 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4ca50-ecd6-48b1-b477-95456254764f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:04.691351 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4ca50-ecd6-48b1-b477-95456254764f admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:04.850546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1012/2034] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:04 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:04.854802 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e2265-0017-40e1-a31d-8494adf445eb admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:04.865925 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e2265-0017-40e1-a31d-8494adf445eb admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:04.947867 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efe7de-320c-4ce3-8594-fed7bdb8cb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:04.955390 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efe7de-320c-4ce3-8594-fed7bdb8cb46 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:04.955390 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efe7de-320c-4ce3-8594-fed7bdb8cb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:04.955390 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efe7de-320c-4ce3-8594-fed7bdb8cb46 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:04.955390 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efe7de-320c-4ce3-8594-fed7bdb8cb46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:04.955390 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efe7de-320c-4ce3-8594-fed7bdb8cb46 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:05.033051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1023/2035] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:04 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:05.037564 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4ca50-ecd6-48b1-b477-95456254764f admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:05.049915 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4ca50-ecd6-48b1-b477-95456254764f admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:05.055848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0841e-2918-42a6-9586-f68e5c2eb84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:05.056657 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0841e-2918-42a6-9586-f68e5c2eb84e admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:05.057106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0841e-2918-42a6-9586-f68e5c2eb84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:05.057457 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0841e-2918-42a6-9586-f68e5c2eb84e admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:05.058599 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0841e-2918-42a6-9586-f68e5c2eb84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:05.060287 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0841e-2918-42a6-9586-f68e5c2eb84e admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:05.072554 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1013/2036] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:18:05.077612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efe7de-320c-4ce3-8594-fed7bdb8cb46 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:05.098518 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efe7de-320c-4ce3-8594-fed7bdb8cb46 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:05.107784 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe37ab-ad20-4756-8a16-5b00e030bb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:05.108975 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe37ab-ad20-4756-8a16-5b00e030bb17 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:05.109416 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe37ab-ad20-4756-8a16-5b00e030bb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:05.109707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe37ab-ad20-4756-8a16-5b00e030bb17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:05.110536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe37ab-ad20-4756-8a16-5b00e030bb17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:05.112728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe37ab-ad20-4756-8a16-5b00e030bb17 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:05.125484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1024/2037] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:18:05.129998 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0841e-2918-42a6-9586-f68e5c2eb84e admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:05.142812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0841e-2918-42a6-9586-f68e5c2eb84e admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:05.148604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ffce7-5678-4df5-9037-5216cda85554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:05.149833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ffce7-5678-4df5-9037-5216cda85554 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:05.150523 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ffce7-5678-4df5-9037-5216cda85554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:05.151247 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ffce7-5678-4df5-9037-5216cda85554 admin admin] PATH_INFO: `/v3/projects/448215ba651142c9a5c2a03ca1951701/users/13e7ca97b2434d218ce6101528965ef0/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:05.170603 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ffce7-5678-4df5-9037-5216cda85554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=448215ba651142c9a5c2a03ca1951701, user_id=13e7ca97b2434d218ce6101528965ef0, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:05.176297 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ffce7-5678-4df5-9037-5216cda85554 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:05.266560 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1014/2038] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:05 2022] PUT /identity/v3/projects/448215ba651142c9a5c2a03ca1951701/users/13e7ca97b2434d218ce6101528965ef0/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:05.276498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e0222-829a-42b5-b829-1af08a1fea36 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:05.276789 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e0222-829a-42b5-b829-1af08a1fea36 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:05.277304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e0222-829a-42b5-b829-1af08a1fea36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:05.312315 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-961e0222-829a-42b5-b829-1af08a1fea36 None None] MFA Rules not processed for user `13e7ca97b2434d218ce6101528965ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:05.313532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-961e0222-829a-42b5-b829-1af08a1fea36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:05.384003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1025/2039] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:05.388531 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ffce7-5678-4df5-9037-5216cda85554 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:05.468489 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ffce7-5678-4df5-9037-5216cda85554 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:05.473832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37391baa-5e9f-4399-b711-266ad1c66830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:05.474534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37391baa-5e9f-4399-b711-266ad1c66830 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:05.474817 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37391baa-5e9f-4399-b711-266ad1c66830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:05.475089 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37391baa-5e9f-4399-b711-266ad1c66830 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:05.475799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37391baa-5e9f-4399-b711-266ad1c66830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:05.477700 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37391baa-5e9f-4399-b711-266ad1c66830 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:05.568750 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e0222-829a-42b5-b829-1af08a1fea36 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:05.570453 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-961e0222-829a-42b5-b829-1af08a1fea36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:05.608531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e0222-829a-42b5-b829-1af08a1fea36 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:05.630547 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf43c9-0fe5-41b5-a6c6-3de9c91b0977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:05.631473 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf43c9-0fe5-41b5-a6c6-3de9c91b0977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:05.631758 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf43c9-0fe5-41b5-a6c6-3de9c91b0977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:05.632038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf43c9-0fe5-41b5-a6c6-3de9c91b0977 admin admin] PATH_INFO: `/v3/users/e1a793e23b934ddb83ea6573fa1eb81a` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:05.633281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf43c9-0fe5-41b5-a6c6-3de9c91b0977 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1a793e23b934ddb83ea6573fa1eb81a)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:05.637941 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf43c9-0fe5-41b5-a6c6-3de9c91b0977 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:05.645450 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1015/2040] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:05 2022] POST /identity/v3/projects => generated 387 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:18:05.660961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37391baa-5e9f-4399-b711-266ad1c66830 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:05.764315 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37391baa-5e9f-4399-b711-266ad1c66830 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:05.783325 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a241ed8-e0a6-43f8-84f8-846e254fa9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:05.784127 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241ed8-e0a6-43f8-84f8-846e254fa9e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:05.784494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241ed8-e0a6-43f8-84f8-846e254fa9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:05.785805 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241ed8-e0a6-43f8-84f8-846e254fa9e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:05.785805 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a241ed8-e0a6-43f8-84f8-846e254fa9e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:05.788045 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a241ed8-e0a6-43f8-84f8-846e254fa9e4 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:05.878144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1016/2041] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:05 2022] POST /identity/v3/users => generated 294 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:05.881793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a241ed8-e0a6-43f8-84f8-846e254fa9e4 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:05.895895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a241ed8-e0a6-43f8-84f8-846e254fa9e4 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:05.904537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-50bf43c9-0fe5-41b5-a6c6-3de9c91b0977 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1a793e23b934ddb83ea6573fa1eb81a'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:05.909019 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-50bf43c9-0fe5-41b5-a6c6-3de9c91b0977 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1a793e23b934ddb83ea6573fa1eb81a'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:05.909019 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-50bf43c9-0fe5-41b5-a6c6-3de9c91b0977 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1a793e23b934ddb83ea6573fa1eb81a'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:05.915353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e5167-764d-4a50-af8c-c0b9f09f87c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:05.916377 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e5167-764d-4a50-af8c-c0b9f09f87c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:05.916862 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e5167-764d-4a50-af8c-c0b9f09f87c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:05.917298 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e5167-764d-4a50-af8c-c0b9f09f87c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:05.918475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e5167-764d-4a50-af8c-c0b9f09f87c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:05.921740 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e5167-764d-4a50-af8c-c0b9f09f87c2 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:05.926571 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-50bf43c9-0fe5-41b5-a6c6-3de9c91b0977 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1a793e23b934ddb83ea6573fa1eb81a'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:05.940571 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1017/2042] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:05.944072 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e5167-764d-4a50-af8c-c0b9f09f87c2 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:05.946633 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-50bf43c9-0fe5-41b5-a6c6-3de9c91b0977 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1a793e23b934ddb83ea6573fa1eb81a'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:05.946792 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1e2e5167-764d-4a50-af8c-c0b9f09f87c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:05.962480 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e5167-764d-4a50-af8c-c0b9f09f87c2 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:05.969363 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb5c63-e027-4199-a629-92b6a343a1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:05.970389 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb5c63-e027-4199-a629-92b6a343a1da admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:05.971039 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb5c63-e027-4199-a629-92b6a343a1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:05.971698 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb5c63-e027-4199-a629-92b6a343a1da admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:05.972713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb5c63-e027-4199-a629-92b6a343a1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:05.975696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb5c63-e027-4199-a629-92b6a343a1da admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:05.994126 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1018/2043] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:05.998488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb5c63-e027-4199-a629-92b6a343a1da admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:06.019156 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb5c63-e027-4199-a629-92b6a343a1da admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:06.023104 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1026/2044] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:05 2022] DELETE /identity/v3/users/e1a793e23b934ddb83ea6573fa1eb81a => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:06.028007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fca30-eea0-4ebd-b652-fad867990a8a None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:06.028303 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fca30-eea0-4ebd-b652-fad867990a8a None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:06.028836 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fca30-eea0-4ebd-b652-fad867990a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:06.048217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-358fca30-eea0-4ebd-b652-fad867990a8a None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:06.051031 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-358fca30-eea0-4ebd-b652-fad867990a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:06.105371 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bce55-c92c-4039-83b5-4b43fecf4919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:06.107164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bce55-c92c-4039-83b5-4b43fecf4919 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:06.107870 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bce55-c92c-4039-83b5-4b43fecf4919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:06.108223 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bce55-c92c-4039-83b5-4b43fecf4919 admin admin] PATH_INFO: `/v3/projects/785547e501014f8ba5db26397d031c8c/users/7e23ecfd75c148bf948940d7fffe51c6/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:06.126893 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1027/2045] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:06.141757 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fca30-eea0-4ebd-b652-fad867990a8a None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:06.146097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-358fca30-eea0-4ebd-b652-fad867990a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:06.149876 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bce55-c92c-4039-83b5-4b43fecf4919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=785547e501014f8ba5db26397d031c8c, user_id=7e23ecfd75c148bf948940d7fffe51c6, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:06.154558 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bce55-c92c-4039-83b5-4b43fecf4919 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:06.255903 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1019/2046] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:05 2022] PUT /identity/v3/projects/785547e501014f8ba5db26397d031c8c/users/7e23ecfd75c148bf948940d7fffe51c6/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:06.266296 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b769f7a-e1b6-418f-ba1c-c20e6d14ac28 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:06.266755 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fca30-eea0-4ebd-b652-fad867990a8a None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:06.267182 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b769f7a-e1b6-418f-ba1c-c20e6d14ac28 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:06.268673 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b769f7a-e1b6-418f-ba1c-c20e6d14ac28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:06.320866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-7b769f7a-e1b6-418f-ba1c-c20e6d14ac28 None None] MFA Rules not processed for user `7e23ecfd75c148bf948940d7fffe51c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:06.322275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-7b769f7a-e1b6-418f-ba1c-c20e6d14ac28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:06.370936 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1fd73-ec70-4dd4-99da-dbb5b2f911a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:06.371725 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1fd73-ec70-4dd4-99da-dbb5b2f911a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:06.372082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1fd73-ec70-4dd4-99da-dbb5b2f911a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:06.372435 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1fd73-ec70-4dd4-99da-dbb5b2f911a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:06.472915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1fd73-ec70-4dd4-99da-dbb5b2f911a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:06.475324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1fd73-ec70-4dd4-99da-dbb5b2f911a1 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:06.494364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1020/2047] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:06.495767 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b769f7a-e1b6-418f-ba1c-c20e6d14ac28 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:06.501093 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1028/2048] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:18:06.526528 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b769f7a-e1b6-418f-ba1c-c20e6d14ac28 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:06.532595 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129bf31-80b9-4599-8bf5-dc45599b26b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:06.533319 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129bf31-80b9-4599-8bf5-dc45599b26b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:06.533575 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129bf31-80b9-4599-8bf5-dc45599b26b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:06.533849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129bf31-80b9-4599-8bf5-dc45599b26b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:06.534831 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129bf31-80b9-4599-8bf5-dc45599b26b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:06.536976 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129bf31-80b9-4599-8bf5-dc45599b26b4 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:06.647532 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1021/2049] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:06 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:06.651555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1fd73-ec70-4dd4-99da-dbb5b2f911a1 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:06.665773 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1fd73-ec70-4dd4-99da-dbb5b2f911a1 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:06.739654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de920c-89d6-4672-8eac-e2af990f6f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:06.740105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de920c-89d6-4672-8eac-e2af990f6f10 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:06.740492 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de920c-89d6-4672-8eac-e2af990f6f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:06.741011 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de920c-89d6-4672-8eac-e2af990f6f10 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:06.746553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de920c-89d6-4672-8eac-e2af990f6f10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:06.746553 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de920c-89d6-4672-8eac-e2af990f6f10 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:06.853746 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1029/2050] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:06 2022] POST /identity/v3/users => generated 300 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:06.860117 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129bf31-80b9-4599-8bf5-dc45599b26b4 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:06.911277 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129bf31-80b9-4599-8bf5-dc45599b26b4 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:06.918143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e309242-71ad-4d9f-a336-3c65a4e3dcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:06.919546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e309242-71ad-4d9f-a336-3c65a4e3dcbf admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:06.919997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e309242-71ad-4d9f-a336-3c65a4e3dcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:06.920398 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e309242-71ad-4d9f-a336-3c65a4e3dcbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:06.921273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e309242-71ad-4d9f-a336-3c65a4e3dcbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:06.923089 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e309242-71ad-4d9f-a336-3c65a4e3dcbf admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:06.942479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de920c-89d6-4672-8eac-e2af990f6f10 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:06.943898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1022/2051] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:06.955853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de920c-89d6-4672-8eac-e2af990f6f10 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:06.960948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e19431-a7f9-45d7-b469-29dc337d3819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:06.961838 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e19431-a7f9-45d7-b469-29dc337d3819 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:06.962139 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e19431-a7f9-45d7-b469-29dc337d3819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:06.962654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e19431-a7f9-45d7-b469-29dc337d3819 admin admin] PATH_INFO: `/v3/projects/b61e26c29698425a99575213eaf1bc46/users/40ee22e521f14608acc89b98ba45002e/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:06.982423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e19431-a7f9-45d7-b469-29dc337d3819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b61e26c29698425a99575213eaf1bc46, user_id=40ee22e521f14608acc89b98ba45002e, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:06.991706 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e19431-a7f9-45d7-b469-29dc337d3819 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:07.107787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e309242-71ad-4d9f-a336-3c65a4e3dcbf admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:07.109368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1030/2052] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:06 2022] PUT /identity/v3/projects/b61e26c29698425a99575213eaf1bc46/users/40ee22e521f14608acc89b98ba45002e/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:07.110313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e19431-a7f9-45d7-b469-29dc337d3819 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:07.111331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-5e309242-71ad-4d9f-a336-3c65a4e3dcbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:07.161212 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e309242-71ad-4d9f-a336-3c65a4e3dcbf admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:07.291933 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e19431-a7f9-45d7-b469-29dc337d3819 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:07.308997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4b9f5-ed9b-488d-a0ad-df71b5a40abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:07.311293 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4b9f5-ed9b-488d-a0ad-df71b5a40abc admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:07.311601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4b9f5-ed9b-488d-a0ad-df71b5a40abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:07.312228 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4b9f5-ed9b-488d-a0ad-df71b5a40abc admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:07.314574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4b9f5-ed9b-488d-a0ad-df71b5a40abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:07.318212 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4b9f5-ed9b-488d-a0ad-df71b5a40abc admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:07.365639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1031/2053] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:07.370252 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4b9f5-ed9b-488d-a0ad-df71b5a40abc admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:07.388826 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1049cdc-c327-49e8-bc7f-f1731ebe5f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:07.389715 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1049cdc-c327-49e8-bc7f-f1731ebe5f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:07.390058 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1049cdc-c327-49e8-bc7f-f1731ebe5f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:07.390346 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1049cdc-c327-49e8-bc7f-f1731ebe5f18 admin admin] PATH_INFO: `/v3/projects/8963f6d764774712a00076158990daa1` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:07.392227 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4b9f5-ed9b-488d-a0ad-df71b5a40abc admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:07.401154 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1049cdc-c327-49e8-bc7f-f1731ebe5f18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8963f6d764774712a00076158990daa1)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:07.403161 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1049cdc-c327-49e8-bc7f-f1731ebe5f18 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:07.414950 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd478f9-ea7f-4fae-9133-8eb980a0eb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:07.421352 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd478f9-ea7f-4fae-9133-8eb980a0eb07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:07.421706 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd478f9-ea7f-4fae-9133-8eb980a0eb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:07.424941 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd478f9-ea7f-4fae-9133-8eb980a0eb07 admin admin] PATH_INFO: `/v3/projects/b61e26c29698425a99575213eaf1bc46/users/40ee22e521f14608acc89b98ba45002e/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:07.435271 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd478f9-ea7f-4fae-9133-8eb980a0eb07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b61e26c29698425a99575213eaf1bc46, user_id=40ee22e521f14608acc89b98ba45002e, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:07.438535 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd478f9-ea7f-4fae-9133-8eb980a0eb07 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:07.466140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1032/2054] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:07 2022] PUT /identity/v3/projects/b61e26c29698425a99575213eaf1bc46/users/40ee22e521f14608acc89b98ba45002e/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:07.484994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2186a-67c0-43ff-904d-fdd13106f73d None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:07.488640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2186a-67c0-43ff-904d-fdd13106f73d None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:07.489130 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2186a-67c0-43ff-904d-fdd13106f73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:07.524211 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-87a2186a-67c0-43ff-904d-fdd13106f73d None None] MFA Rules not processed for user `40ee22e521f14608acc89b98ba45002e`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:07.525720 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-87a2186a-67c0-43ff-904d-fdd13106f73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:07.573242 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-a1049cdc-c327-49e8-bc7f-f1731ebe5f18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8963f6d764774712a00076158990daa1', 'request_id': 'req-a1049cdc-c327-49e8-bc7f-f1731ebe5f18'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:07.640700 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1033/2055] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:07.669631 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-a1049cdc-c327-49e8-bc7f-f1731ebe5f18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8963f6d764774712a00076158990daa1', 'request_id': 'req-a1049cdc-c327-49e8-bc7f-f1731ebe5f18'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:07.706457 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2186a-67c0-43ff-904d-fdd13106f73d None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:07.707295 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-a1049cdc-c327-49e8-bc7f-f1731ebe5f18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8963f6d764774712a00076158990daa1', 'request_id': 'req-a1049cdc-c327-49e8-bc7f-f1731ebe5f18'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:07.707403 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-87a2186a-67c0-43ff-904d-fdd13106f73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:07.709205 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-a1049cdc-c327-49e8-bc7f-f1731ebe5f18 admin admin] The token cache is being invalidate because project 8963f6d764774712a00076158990daa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:07.709587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-a1049cdc-c327-49e8-bc7f-f1731ebe5f18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:07.710404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-a1049cdc-c327-49e8-bc7f-f1731ebe5f18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:07.715663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1023/2056] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:07 2022] DELETE /identity/v3/projects/8963f6d764774712a00076158990daa1 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:07.717473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1049cdc-c327-49e8-bc7f-f1731ebe5f18 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:07.719148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a1049cdc-c327-49e8-bc7f-f1731ebe5f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:07.782398 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2186a-67c0-43ff-904d-fdd13106f73d None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:07.811923 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1049cdc-c327-49e8-bc7f-f1731ebe5f18 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:07.825187 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f57c2-4ce7-421f-8404-8424e4c459d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:07.826716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f57c2-4ce7-421f-8404-8424e4c459d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:07.826993 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f57c2-4ce7-421f-8404-8424e4c459d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:07.828303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f57c2-4ce7-421f-8404-8424e4c459d6 admin admin] PATH_INFO: `/v3/users/194ce1f9b8b441f4bbbe8d51bb800ec6` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:07.831519 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f57c2-4ce7-421f-8404-8424e4c459d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=194ce1f9b8b441f4bbbe8d51bb800ec6)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:07.833481 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f57c2-4ce7-421f-8404-8424e4c459d6 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:07.848987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6c1ab-6f55-4cfd-b6f1-81d8eaa95283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:07.849930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6c1ab-6f55-4cfd-b6f1-81d8eaa95283 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:07.850359 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6c1ab-6f55-4cfd-b6f1-81d8eaa95283 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:07.850731 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6c1ab-6f55-4cfd-b6f1-81d8eaa95283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:07.857450 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-39b6c1ab-6f55-4cfd-b6f1-81d8eaa95283 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:07.944427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6c1ab-6f55-4cfd-b6f1-81d8eaa95283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:07.946039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6c1ab-6f55-4cfd-b6f1-81d8eaa95283 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:07.957832 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1034/2057] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:18:08.074980 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-fc8f57c2-4ce7-421f-8404-8424e4c459d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '194ce1f9b8b441f4bbbe8d51bb800ec6'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:08.075670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-fc8f57c2-4ce7-421f-8404-8424e4c459d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '194ce1f9b8b441f4bbbe8d51bb800ec6'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:08.099925 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-fc8f57c2-4ce7-421f-8404-8424e4c459d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '194ce1f9b8b441f4bbbe8d51bb800ec6'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:08.100675 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-fc8f57c2-4ce7-421f-8404-8424e4c459d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '194ce1f9b8b441f4bbbe8d51bb800ec6'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:08.108604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-fc8f57c2-4ce7-421f-8404-8424e4c459d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '194ce1f9b8b441f4bbbe8d51bb800ec6'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:08.171888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1024/2058] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:07 2022] DELETE /identity/v3/users/194ce1f9b8b441f4bbbe8d51bb800ec6 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:09.242020 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6c1ab-6f55-4cfd-b6f1-81d8eaa95283 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:09.256086 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6c1ab-6f55-4cfd-b6f1-81d8eaa95283 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:09.390531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9ed4e-f136-4536-b57e-9744553f7760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:09.391405 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9ed4e-f136-4536-b57e-9744553f7760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:09.391691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9ed4e-f136-4536-b57e-9744553f7760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:09.391967 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9ed4e-f136-4536-b57e-9744553f7760 admin admin] PATH_INFO: `/v3/projects/41cb7b51ac4a449e81af84abff566bcd` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:09.394178 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9ed4e-f136-4536-b57e-9744553f7760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41cb7b51ac4a449e81af84abff566bcd)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:09.396086 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9ed4e-f136-4536-b57e-9744553f7760 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:09.401539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f57c2-4ce7-421f-8404-8424e4c459d6 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:09.402980 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-fc8f57c2-4ce7-421f-8404-8424e4c459d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:09.490075 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f57c2-4ce7-421f-8404-8424e4c459d6 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:09.494547 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398d718-7930-48e3-aedb-02dde8584b1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:09.495288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398d718-7930-48e3-aedb-02dde8584b1b service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:09.495669 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398d718-7930-48e3-aedb-02dde8584b1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:09.496021 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398d718-7930-48e3-aedb-02dde8584b1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:09.498787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-5398d718-7930-48e3-aedb-02dde8584b1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:09.585428 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398d718-7930-48e3-aedb-02dde8584b1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:09.590765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398d718-7930-48e3-aedb-02dde8584b1b service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:09.604847 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-59f9ed4e-f136-4536-b57e-9744553f7760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41cb7b51ac4a449e81af84abff566bcd', 'request_id': 'req-59f9ed4e-f136-4536-b57e-9744553f7760'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:09.616107 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1025/2059] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:18:09.701397 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-59f9ed4e-f136-4536-b57e-9744553f7760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41cb7b51ac4a449e81af84abff566bcd', 'request_id': 'req-59f9ed4e-f136-4536-b57e-9744553f7760'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:09.710827 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-59f9ed4e-f136-4536-b57e-9744553f7760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41cb7b51ac4a449e81af84abff566bcd', 'request_id': 'req-59f9ed4e-f136-4536-b57e-9744553f7760'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:09.722960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-59f9ed4e-f136-4536-b57e-9744553f7760 admin admin] The token cache is being invalidate because project 41cb7b51ac4a449e81af84abff566bcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:09.723330 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-59f9ed4e-f136-4536-b57e-9744553f7760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:09.724072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-59f9ed4e-f136-4536-b57e-9744553f7760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:09.726019 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1035/2060] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:09 2022] DELETE /identity/v3/projects/41cb7b51ac4a449e81af84abff566bcd => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:09.731712 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502adb13-4729-47e2-969a-0e595b31c666 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:09.732154 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502adb13-4729-47e2-969a-0e595b31c666 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:09.732705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502adb13-4729-47e2-969a-0e595b31c666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:09.759733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-502adb13-4729-47e2-969a-0e595b31c666 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:09.762025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-502adb13-4729-47e2-969a-0e595b31c666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:09.828843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1026/2061] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:09.836720 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e2ee7-941e-44ed-bcda-68abcf5eeac2 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:09.837004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e2ee7-941e-44ed-bcda-68abcf5eeac2 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:09.837282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e2ee7-941e-44ed-bcda-68abcf5eeac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:09.856400 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-026e2ee7-941e-44ed-bcda-68abcf5eeac2 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:09.857843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-026e2ee7-941e-44ed-bcda-68abcf5eeac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:09.866852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1036/2062] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:09.871809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502adb13-4729-47e2-969a-0e595b31c666 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:09.883524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502adb13-4729-47e2-969a-0e595b31c666 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:09.890838 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff38102-ba70-4a7f-bd79-281de8df6540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:09.891519 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff38102-ba70-4a7f-bd79-281de8df6540 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:09.891802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff38102-ba70-4a7f-bd79-281de8df6540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:09.892126 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff38102-ba70-4a7f-bd79-281de8df6540 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:09.892918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff38102-ba70-4a7f-bd79-281de8df6540 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:18:09.893355 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff38102-ba70-4a7f-bd79-281de8df6540 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:09.902654 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff38102-ba70-4a7f-bd79-281de8df6540 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:09.924343 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1027/2063] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 20:18:09.926319 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e2ee7-941e-44ed-bcda-68abcf5eeac2 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:09.936958 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e2ee7-941e-44ed-bcda-68abcf5eeac2 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:09.942398 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457e03c-82b8-4f9d-a7d2-ce63360e88a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:09.943046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457e03c-82b8-4f9d-a7d2-ce63360e88a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:09.943325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457e03c-82b8-4f9d-a7d2-ce63360e88a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:09.943595 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457e03c-82b8-4f9d-a7d2-ce63360e88a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:09.944295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457e03c-82b8-4f9d-a7d2-ce63360e88a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:09.946264 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457e03c-82b8-4f9d-a7d2-ce63360e88a8 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:10.069763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1037/2064] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:09 2022] POST /identity/v3/projects => generated 397 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:10.070028 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff38102-ba70-4a7f-bd79-281de8df6540 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:10.088145 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff38102-ba70-4a7f-bd79-281de8df6540 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:10.238448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee4e7c-77e6-47c9-9b97-dd2c0bcca952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:10.239246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee4e7c-77e6-47c9-9b97-dd2c0bcca952 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:10.239632 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee4e7c-77e6-47c9-9b97-dd2c0bcca952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:10.240012 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee4e7c-77e6-47c9-9b97-dd2c0bcca952 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:10.240834 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee4e7c-77e6-47c9-9b97-dd2c0bcca952 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:10.243226 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee4e7c-77e6-47c9-9b97-dd2c0bcca952 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:10.335467 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457e03c-82b8-4f9d-a7d2-ce63360e88a8 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:10.342728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1028/2065] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:10 2022] POST /identity/v3/users => generated 299 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:18:10.347743 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457e03c-82b8-4f9d-a7d2-ce63360e88a8 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:10.356677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fb4b5-01cf-4b53-b471-a9ad646f1aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:10.357636 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fb4b5-01cf-4b53-b471-a9ad646f1aef admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:10.358124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fb4b5-01cf-4b53-b471-a9ad646f1aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:10.359027 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fb4b5-01cf-4b53-b471-a9ad646f1aef admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:10.360285 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fb4b5-01cf-4b53-b471-a9ad646f1aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:10.362937 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fb4b5-01cf-4b53-b471-a9ad646f1aef admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:10.378337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1038/2066] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:18:10.382385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee4e7c-77e6-47c9-9b97-dd2c0bcca952 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:10.395256 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee4e7c-77e6-47c9-9b97-dd2c0bcca952 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:10.400862 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ebf8d-aee3-4d4e-850b-b10e5fd57bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:10.401728 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ebf8d-aee3-4d4e-850b-b10e5fd57bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:10.402785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ebf8d-aee3-4d4e-850b-b10e5fd57bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:10.403203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ebf8d-aee3-4d4e-850b-b10e5fd57bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:10.404101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ebf8d-aee3-4d4e-850b-b10e5fd57bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:10.405880 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ebf8d-aee3-4d4e-850b-b10e5fd57bf3 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:10.433389 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fb4b5-01cf-4b53-b471-a9ad646f1aef admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:10.443350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1029/2067] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:10.447455 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fb4b5-01cf-4b53-b471-a9ad646f1aef admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:10.465915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4846f7-932d-4bfb-b63d-7530644befdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:10.465915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4846f7-932d-4bfb-b63d-7530644befdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:10.466234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4846f7-932d-4bfb-b63d-7530644befdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:10.466522 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4846f7-932d-4bfb-b63d-7530644befdf admin admin] PATH_INFO: `/v3/projects/cd91e5efbc94497a90a7a4988669d653/users/7f1310f889694e18b118dd32ef0d5626/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:10.510063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4846f7-932d-4bfb-b63d-7530644befdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd91e5efbc94497a90a7a4988669d653, user_id=7f1310f889694e18b118dd32ef0d5626, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:10.512969 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4846f7-932d-4bfb-b63d-7530644befdf admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:10.645063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1039/2068] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:10 2022] PUT /identity/v3/projects/cd91e5efbc94497a90a7a4988669d653/users/7f1310f889694e18b118dd32ef0d5626/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:10.664786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796696ff-338e-4362-9c8e-491596d17d42 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:10.664786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796696ff-338e-4362-9c8e-491596d17d42 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:10.664786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796696ff-338e-4362-9c8e-491596d17d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:10.725341 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-796696ff-338e-4362-9c8e-491596d17d42 None None] MFA Rules not processed for user `7f1310f889694e18b118dd32ef0d5626`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:10.727659 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-796696ff-338e-4362-9c8e-491596d17d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:10.870701 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1030/2069] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:18:10.875623 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4846f7-932d-4bfb-b63d-7530644befdf admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:10.953032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4846f7-932d-4bfb-b63d-7530644befdf admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:10.958545 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc01f0-7410-4bff-a6ad-0e1031b97885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:10.959222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc01f0-7410-4bff-a6ad-0e1031b97885 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:10.962466 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc01f0-7410-4bff-a6ad-0e1031b97885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:10.962466 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc01f0-7410-4bff-a6ad-0e1031b97885 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:10.962466 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc01f0-7410-4bff-a6ad-0e1031b97885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:10.962466 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc01f0-7410-4bff-a6ad-0e1031b97885 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:11.039829 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1040/2070] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:10 2022] POST /identity/v3/projects => generated 401 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:11.044181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796696ff-338e-4362-9c8e-491596d17d42 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:11.056565 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796696ff-338e-4362-9c8e-491596d17d42 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:11.137677 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716571a4-5c7e-4340-a481-1a38d8cbe18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:11.138398 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716571a4-5c7e-4340-a481-1a38d8cbe18a admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:11.138685 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716571a4-5c7e-4340-a481-1a38d8cbe18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:11.138987 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716571a4-5c7e-4340-a481-1a38d8cbe18a admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:11.139700 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716571a4-5c7e-4340-a481-1a38d8cbe18a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:11.141613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716571a4-5c7e-4340-a481-1a38d8cbe18a admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:11.223196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1031/2071] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:11 2022] POST /identity/v3/users => generated 307 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:11.225013 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc01f0-7410-4bff-a6ad-0e1031b97885 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:11.240630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc01f0-7410-4bff-a6ad-0e1031b97885 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:11.245970 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649c79-1e40-4298-8e50-90bff05f3638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:11.246670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649c79-1e40-4298-8e50-90bff05f3638 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:11.247119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649c79-1e40-4298-8e50-90bff05f3638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:11.247540 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649c79-1e40-4298-8e50-90bff05f3638 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:11.248818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0649c79-1e40-4298-8e50-90bff05f3638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:11.250797 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0649c79-1e40-4298-8e50-90bff05f3638 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:11.277513 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1041/2072] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:11.279766 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716571a4-5c7e-4340-a481-1a38d8cbe18a admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:11.290539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716571a4-5c7e-4340-a481-1a38d8cbe18a admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:11.295504 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4d0e1-9c32-4753-badf-ef29c3136a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:11.296412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4d0e1-9c32-4753-badf-ef29c3136a7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:11.296788 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4d0e1-9c32-4753-badf-ef29c3136a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:11.297094 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4d0e1-9c32-4753-badf-ef29c3136a7c admin admin] PATH_INFO: `/v3/projects/cd8ea61cf687437e96faff4638cd3a48/users/7c56c23fc59646f7be726407dfccb83d/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:11.321012 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4d0e1-9c32-4753-badf-ef29c3136a7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8ea61cf687437e96faff4638cd3a48, user_id=7c56c23fc59646f7be726407dfccb83d, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:11.325012 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4d0e1-9c32-4753-badf-ef29c3136a7c admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:11.405060 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1032/2073] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:11 2022] PUT /identity/v3/projects/cd8ea61cf687437e96faff4638cd3a48/users/7c56c23fc59646f7be726407dfccb83d/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:11.409702 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649c79-1e40-4298-8e50-90bff05f3638 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:11.427068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649c79-1e40-4298-8e50-90bff05f3638 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:11.500389 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a6361-5fc5-4477-9875-0d6d1c0d45ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:11.501055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a6361-5fc5-4477-9875-0d6d1c0d45ca admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:11.501340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a6361-5fc5-4477-9875-0d6d1c0d45ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:11.501616 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a6361-5fc5-4477-9875-0d6d1c0d45ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:11.502385 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a6361-5fc5-4477-9875-0d6d1c0d45ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:11.504571 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a6361-5fc5-4477-9875-0d6d1c0d45ca admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:11.514305 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1042/2074] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:11.518489 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4d0e1-9c32-4753-badf-ef29c3136a7c admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:11.528799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4d0e1-9c32-4753-badf-ef29c3136a7c admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:11.533313 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519cd04-a756-48ba-9eab-8102248c753f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:11.534325 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519cd04-a756-48ba-9eab-8102248c753f admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:11.534606 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519cd04-a756-48ba-9eab-8102248c753f admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:11.534877 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519cd04-a756-48ba-9eab-8102248c753f admin admin] PATH_INFO: `/v3/projects/cd8ea61cf687437e96faff4638cd3a48/users/7c56c23fc59646f7be726407dfccb83d/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:11.536833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c519cd04-a756-48ba-9eab-8102248c753f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8ea61cf687437e96faff4638cd3a48, user_id=7c56c23fc59646f7be726407dfccb83d, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:11.539512 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c519cd04-a756-48ba-9eab-8102248c753f admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:11.552984 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1033/2075] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:11 2022] PUT /identity/v3/projects/cd8ea61cf687437e96faff4638cd3a48/users/7c56c23fc59646f7be726407dfccb83d/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:11.554329 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf17379-ed0b-49be-aa68-9e1037438816 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:11.554625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf17379-ed0b-49be-aa68-9e1037438816 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:11.555069 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf17379-ed0b-49be-aa68-9e1037438816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:11.563143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83cd93-9f0c-4ff0-98e9-de7e47224f10 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:11.563428 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83cd93-9f0c-4ff0-98e9-de7e47224f10 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:11.563691 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83cd93-9f0c-4ff0-98e9-de7e47224f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:11.584994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-6cf17379-ed0b-49be-aa68-9e1037438816 None None] MFA Rules not processed for user `72111ba468f744b2a50b100e15f6c7f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:11.592160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-6cf17379-ed0b-49be-aa68-9e1037438816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:11.600455 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-5f83cd93-9f0c-4ff0-98e9-de7e47224f10 None None] MFA Rules not processed for user `7c56c23fc59646f7be726407dfccb83d`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:11.601978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-5f83cd93-9f0c-4ff0-98e9-de7e47224f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:11.655806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1043/2076] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:18:11.668534 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf17379-ed0b-49be-aa68-9e1037438816 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:11.669886 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-6cf17379-ed0b-49be-aa68-9e1037438816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:11.671475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1034/2077] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:11 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:11.682943 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83cd93-9f0c-4ff0-98e9-de7e47224f10 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:11.684587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-5f83cd93-9f0c-4ff0-98e9-de7e47224f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:11.726397 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf17379-ed0b-49be-aa68-9e1037438816 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:11.729993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638974c-c78f-4a6a-a645-6ea80af111be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67a252cde71a455fb1dc4bf9c47c0490', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:11.730666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638974c-c78f-4a6a-a645-6ea80af111be service glance] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:11.730950 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638974c-c78f-4a6a-a645-6ea80af111be service glance] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:11.731231 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638974c-c78f-4a6a-a645-6ea80af111be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:11.733042 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b638974c-c78f-4a6a-a645-6ea80af111be service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:11.740490 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83cd93-9f0c-4ff0-98e9-de7e47224f10 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:11.744224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f169595-0af2-491f-850b-669ec33376ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:11.744882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f169595-0af2-491f-850b-669ec33376ba service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:11.745165 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f169595-0af2-491f-850b-669ec33376ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:11.745430 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f169595-0af2-491f-850b-669ec33376ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:11.754939 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f169595-0af2-491f-850b-669ec33376ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:11.756405 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f169595-0af2-491f-850b-669ec33376ba service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:11.765698 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1035/2078] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:11 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:18:11.791251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638974c-c78f-4a6a-a645-6ea80af111be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:11.792607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638974c-c78f-4a6a-a645-6ea80af111be service glance] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:11.801518 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1044/2079] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:18:11.961541 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f169595-0af2-491f-850b-669ec33376ba service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:11.970307 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f169595-0af2-491f-850b-669ec33376ba service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:11.974434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f5bd0-1b3f-4cca-a108-438be2fe303a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:11.983469 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f5bd0-1b3f-4cca-a108-438be2fe303a service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:11.983469 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f5bd0-1b3f-4cca-a108-438be2fe303a service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:11.983469 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f5bd0-1b3f-4cca-a108-438be2fe303a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:12.024834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8fb5c-7ca2-4c75-a209-5bbd1e671020 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:12.025165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8fb5c-7ca2-4c75-a209-5bbd1e671020 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:12.025467 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8fb5c-7ca2-4c75-a209-5bbd1e671020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:12.037246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f5bd0-1b3f-4cca-a108-438be2fe303a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:12.038733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f5bd0-1b3f-4cca-a108-438be2fe303a service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:12.044375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-10f8fb5c-7ca2-4c75-a209-5bbd1e671020 None None] MFA Rules not processed for user `7ec2605fd1b84e0ea134b943ed33a605`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:12.045550 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-10f8fb5c-7ca2-4c75-a209-5bbd1e671020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:12.049347 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1036/2080] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:12.111892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1045/2081] 10.222.0.65 () {58 vars in 1009 bytes} [Tue May 10 20:18:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:18:12.163687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f5bd0-1b3f-4cca-a108-438be2fe303a service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:12.165615 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1a5f5bd0-1b3f-4cca-a108-438be2fe303a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:12.262166 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f5bd0-1b3f-4cca-a108-438be2fe303a service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:12.271559 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf809b81-d355-4e68-aba5-cc62ae75f09e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d10b258a53484da22950e2b1db2e41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:12.272771 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf809b81-d355-4e68-aba5-cc62ae75f09e service swift] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:12.272771 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf809b81-d355-4e68-aba5-cc62ae75f09e service swift] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:12.272771 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf809b81-d355-4e68-aba5-cc62ae75f09e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:12.282546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf809b81-d355-4e68-aba5-cc62ae75f09e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:12.284182 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf809b81-d355-4e68-aba5-cc62ae75f09e service swift] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:12.293118 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1037/2082] 10.222.0.65 () {60 vars in 1384 bytes} [Tue May 10 20:18:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 20:18:18.707397 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8fb5c-7ca2-4c75-a209-5bbd1e671020 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:18.709443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-10f8fb5c-7ca2-4c75-a209-5bbd1e671020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:18.733172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8fb5c-7ca2-4c75-a209-5bbd1e671020 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:18.739229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db36e19-5945-4232-aed2-b163be75f276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:18.740504 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db36e19-5945-4232-aed2-b163be75f276 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:18.740804 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db36e19-5945-4232-aed2-b163be75f276 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:18.741095 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db36e19-5945-4232-aed2-b163be75f276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:18.745252 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3db36e19-5945-4232-aed2-b163be75f276 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:18.845276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db36e19-5945-4232-aed2-b163be75f276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:18.846385 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db36e19-5945-4232-aed2-b163be75f276 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:18.859751 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1046/2083] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:19.844891 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf809b81-d355-4e68-aba5-cc62ae75f09e service swift] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:19.846427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-bf809b81-d355-4e68-aba5-cc62ae75f09e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:19.966513 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf809b81-d355-4e68-aba5-cc62ae75f09e service swift] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:19.979991 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802943b-e173-4a4f-829d-54fca2893ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:19.980909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802943b-e173-4a4f-829d-54fca2893ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:19.981186 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802943b-e173-4a4f-829d-54fca2893ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:19.981469 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802943b-e173-4a4f-829d-54fca2893ca7 admin admin] PATH_INFO: `/v3/users/cc740d5c78b348b8a8497a78560a909c` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:19.982761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802943b-e173-4a4f-829d-54fca2893ca7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc740d5c78b348b8a8497a78560a909c)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:19.985236 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802943b-e173-4a4f-829d-54fca2893ca7 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:20.057235 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db36e19-5945-4232-aed2-b163be75f276 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:20.074185 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db36e19-5945-4232-aed2-b163be75f276 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:20.085766 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c34c3-4629-4ee8-872b-5f5f740b8a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:20.087252 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c34c3-4629-4ee8-872b-5f5f740b8a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:20.087683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c34c3-4629-4ee8-872b-5f5f740b8a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:20.088095 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c34c3-4629-4ee8-872b-5f5f740b8a87 admin admin] PATH_INFO: `/v3/users/7f1310f889694e18b118dd32ef0d5626` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:20.089802 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c34c3-4629-4ee8-872b-5f5f740b8a87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f1310f889694e18b118dd32ef0d5626)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:20.092479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c34c3-4629-4ee8-872b-5f5f740b8a87 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:20.330648 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f802943b-e173-4a4f-829d-54fca2893ca7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc740d5c78b348b8a8497a78560a909c'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:20.332131 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f802943b-e173-4a4f-829d-54fca2893ca7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc740d5c78b348b8a8497a78560a909c'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:20.359016 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f802943b-e173-4a4f-829d-54fca2893ca7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc740d5c78b348b8a8497a78560a909c'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:20.360837 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f802943b-e173-4a4f-829d-54fca2893ca7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc740d5c78b348b8a8497a78560a909c'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:20.360999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ba5c34c3-4629-4ee8-872b-5f5f740b8a87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f1310f889694e18b118dd32ef0d5626'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:20.361965 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ba5c34c3-4629-4ee8-872b-5f5f740b8a87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f1310f889694e18b118dd32ef0d5626'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:20.363342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ba5c34c3-4629-4ee8-872b-5f5f740b8a87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f1310f889694e18b118dd32ef0d5626'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:20.381162 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f802943b-e173-4a4f-829d-54fca2893ca7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc740d5c78b348b8a8497a78560a909c'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:20.382045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ba5c34c3-4629-4ee8-872b-5f5f740b8a87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f1310f889694e18b118dd32ef0d5626'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:20.389686 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ba5c34c3-4629-4ee8-872b-5f5f740b8a87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f1310f889694e18b118dd32ef0d5626'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:20.464515 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1038/2084] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:19 2022] DELETE /identity/v3/users/cc740d5c78b348b8a8497a78560a909c => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:20.469099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59063dda-40aa-4979-8175-797822397566 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:20.469385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59063dda-40aa-4979-8175-797822397566 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:20.470043 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59063dda-40aa-4979-8175-797822397566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:20.490379 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-59063dda-40aa-4979-8175-797822397566 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:20.491665 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-59063dda-40aa-4979-8175-797822397566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:20.537627 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1047/2085] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:20 2022] DELETE /identity/v3/users/7f1310f889694e18b118dd32ef0d5626 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:20.544188 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab58fd9-ca5f-4a69-a112-961a4255a0cb None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:20.544765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab58fd9-ca5f-4a69-a112-961a4255a0cb None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:20.544765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab58fd9-ca5f-4a69-a112-961a4255a0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:20.564232 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-dab58fd9-ca5f-4a69-a112-961a4255a0cb None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:20.565517 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-dab58fd9-ca5f-4a69-a112-961a4255a0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:20.604942 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1039/2086] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:20.615731 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59063dda-40aa-4979-8175-797822397566 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:20.617166 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-59063dda-40aa-4979-8175-797822397566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:20.687276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1048/2087] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:20.700233 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab58fd9-ca5f-4a69-a112-961a4255a0cb None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:20.701732 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-dab58fd9-ca5f-4a69-a112-961a4255a0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:20.721959 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59063dda-40aa-4979-8175-797822397566 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:20.782041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89d7e6-91e6-4a0f-862c-f7a14246abb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:20.783008 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89d7e6-91e6-4a0f-862c-f7a14246abb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:20.783440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89d7e6-91e6-4a0f-862c-f7a14246abb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:20.783723 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89d7e6-91e6-4a0f-862c-f7a14246abb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:20.805108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89d7e6-91e6-4a0f-862c-f7a14246abb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:20.806613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89d7e6-91e6-4a0f-862c-f7a14246abb2 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:20.808072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab58fd9-ca5f-4a69-a112-961a4255a0cb None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:20.812226 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbad948-61d8-4ff5-96df-c1162509897f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:20.813401 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbad948-61d8-4ff5-96df-c1162509897f service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:20.813784 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbad948-61d8-4ff5-96df-c1162509897f service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:20.814230 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbad948-61d8-4ff5-96df-c1162509897f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:20.820921 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1040/2088] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:20.826303 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbad948-61d8-4ff5-96df-c1162509897f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:20.828040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbad948-61d8-4ff5-96df-c1162509897f service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:20.842121 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1049/2089] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:21.474546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89d7e6-91e6-4a0f-862c-f7a14246abb2 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:21.478965 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-3e89d7e6-91e6-4a0f-862c-f7a14246abb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:21.500242 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbad948-61d8-4ff5-96df-c1162509897f service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:21.502747 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-8bbad948-61d8-4ff5-96df-c1162509897f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:21.521436 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbad948-61d8-4ff5-96df-c1162509897f service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:21.535279 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f81b7-c82b-4c19-938f-cdd801ae64ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:21.541352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89d7e6-91e6-4a0f-862c-f7a14246abb2 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:21.549175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba249c82-7219-453d-8f27-d8966472622c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:21.553538 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba249c82-7219-453d-8f27-d8966472622c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:21.558056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f81b7-c82b-4c19-938f-cdd801ae64ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:21.558056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f81b7-c82b-4c19-938f-cdd801ae64ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:21.558056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f81b7-c82b-4c19-938f-cdd801ae64ff admin admin] PATH_INFO: `/v3/projects/cd91e5efbc94497a90a7a4988669d653` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:21.558959 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba249c82-7219-453d-8f27-d8966472622c admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:21.558959 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba249c82-7219-453d-8f27-d8966472622c admin admin] PATH_INFO: `/v3/projects/8c0ca727fd7f4a86b103933126caae5e` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:21.559289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f81b7-c82b-4c19-938f-cdd801ae64ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd91e5efbc94497a90a7a4988669d653)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:21.560106 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f81b7-c82b-4c19-938f-cdd801ae64ff admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:21.561038 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba249c82-7219-453d-8f27-d8966472622c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c0ca727fd7f4a86b103933126caae5e)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:21.565424 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba249c82-7219-453d-8f27-d8966472622c admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:21.713907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-ba249c82-7219-453d-8f27-d8966472622c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c0ca727fd7f4a86b103933126caae5e', 'request_id': 'req-ba249c82-7219-453d-8f27-d8966472622c'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:21.743119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8d1f81b7-c82b-4c19-938f-cdd801ae64ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd91e5efbc94497a90a7a4988669d653', 'request_id': 'req-8d1f81b7-c82b-4c19-938f-cdd801ae64ff'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:21.808418 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-ba249c82-7219-453d-8f27-d8966472622c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c0ca727fd7f4a86b103933126caae5e', 'request_id': 'req-ba249c82-7219-453d-8f27-d8966472622c'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:21.816435 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-ba249c82-7219-453d-8f27-d8966472622c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c0ca727fd7f4a86b103933126caae5e', 'request_id': 'req-ba249c82-7219-453d-8f27-d8966472622c'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:21.828497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-ba249c82-7219-453d-8f27-d8966472622c admin admin] The token cache is being invalidate because project 8c0ca727fd7f4a86b103933126caae5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:21.829030 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-ba249c82-7219-453d-8f27-d8966472622c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:21.830416 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-ba249c82-7219-453d-8f27-d8966472622c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:21.832252 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1041/2090] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:21 2022] DELETE /identity/v3/projects/8c0ca727fd7f4a86b103933126caae5e => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:21.837455 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30e10e-c29c-4e1c-a6db-3e6e9144404c None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:21.840542 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30e10e-c29c-4e1c-a6db-3e6e9144404c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:21.840542 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30e10e-c29c-4e1c-a6db-3e6e9144404c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:21.862350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-1a30e10e-c29c-4e1c-a6db-3e6e9144404c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:21.863882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1a30e10e-c29c-4e1c-a6db-3e6e9144404c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:21.888782 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8d1f81b7-c82b-4c19-938f-cdd801ae64ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd91e5efbc94497a90a7a4988669d653', 'request_id': 'req-8d1f81b7-c82b-4c19-938f-cdd801ae64ff'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:21.895673 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8d1f81b7-c82b-4c19-938f-cdd801ae64ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd91e5efbc94497a90a7a4988669d653', 'request_id': 'req-8d1f81b7-c82b-4c19-938f-cdd801ae64ff'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:21.916354 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8d1f81b7-c82b-4c19-938f-cdd801ae64ff admin admin] The token cache is being invalidate because project cd91e5efbc94497a90a7a4988669d653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:21.916843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8d1f81b7-c82b-4c19-938f-cdd801ae64ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:21.917622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8d1f81b7-c82b-4c19-938f-cdd801ae64ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:21.919820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1050/2091] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:21 2022] DELETE /identity/v3/projects/cd91e5efbc94497a90a7a4988669d653 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:21.928443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f81b7-c82b-4c19-938f-cdd801ae64ff admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:21.930460 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-8d1f81b7-c82b-4c19-938f-cdd801ae64ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:21.955002 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1042/2092] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:18:21.968575 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b55051-d35e-43bd-8cae-8a70f466438d None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:21.968803 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b55051-d35e-43bd-8cae-8a70f466438d None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:21.969087 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b55051-d35e-43bd-8cae-8a70f466438d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:22.005090 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-11b55051-d35e-43bd-8cae-8a70f466438d None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:22.007078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-11b55051-d35e-43bd-8cae-8a70f466438d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:22.024284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1043/2093] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:22.025137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b55051-d35e-43bd-8cae-8a70f466438d None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:22.030422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f81b7-c82b-4c19-938f-cdd801ae64ff admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:22.043469 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b55051-d35e-43bd-8cae-8a70f466438d None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:22.044250 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211385ce-e068-406e-8ffa-7f4c32884975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:22.045440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211385ce-e068-406e-8ffa-7f4c32884975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:22.045882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211385ce-e068-406e-8ffa-7f4c32884975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:22.046376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211385ce-e068-406e-8ffa-7f4c32884975 admin admin] PATH_INFO: `/v3/users/7c56c23fc59646f7be726407dfccb83d` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:22.048094 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211385ce-e068-406e-8ffa-7f4c32884975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c56c23fc59646f7be726407dfccb83d)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:22.048830 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdceb39-37e3-4861-aafe-d971dc3527d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:22.050699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdceb39-37e3-4861-aafe-d971dc3527d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:22.051085 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211385ce-e068-406e-8ffa-7f4c32884975 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:22.051746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdceb39-37e3-4861-aafe-d971dc3527d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:22.053333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdceb39-37e3-4861-aafe-d971dc3527d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:22.054136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdceb39-37e3-4861-aafe-d971dc3527d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:18:22.054453 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdceb39-37e3-4861-aafe-d971dc3527d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:22.059408 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdceb39-37e3-4861-aafe-d971dc3527d6 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:22.099069 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1044/2094] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:18:22.100289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdceb39-37e3-4861-aafe-d971dc3527d6 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:22.116304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdceb39-37e3-4861-aafe-d971dc3527d6 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:22.123114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578ad59-016f-450f-b66c-5ba567597de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:22.123837 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578ad59-016f-450f-b66c-5ba567597de5 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:22.124150 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578ad59-016f-450f-b66c-5ba567597de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:22.124431 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578ad59-016f-450f-b66c-5ba567597de5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:22.125159 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2578ad59-016f-450f-b66c-5ba567597de5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:22.127109 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2578ad59-016f-450f-b66c-5ba567597de5 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:22.243826 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1045/2095] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:22 2022] POST /identity/v3/projects => generated 375 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:18:22.249480 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578ad59-016f-450f-b66c-5ba567597de5 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:22.279195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-211385ce-e068-406e-8ffa-7f4c32884975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c56c23fc59646f7be726407dfccb83d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:22.279992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-211385ce-e068-406e-8ffa-7f4c32884975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c56c23fc59646f7be726407dfccb83d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:22.280702 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-211385ce-e068-406e-8ffa-7f4c32884975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c56c23fc59646f7be726407dfccb83d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:22.295748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-211385ce-e068-406e-8ffa-7f4c32884975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c56c23fc59646f7be726407dfccb83d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:22.307814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-211385ce-e068-406e-8ffa-7f4c32884975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c56c23fc59646f7be726407dfccb83d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:22.357373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578ad59-016f-450f-b66c-5ba567597de5 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:22.362937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ca356-3659-4426-a8db-f7a9e52afe87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:22.363698 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ca356-3659-4426-a8db-f7a9e52afe87 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:22.364055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ca356-3659-4426-a8db-f7a9e52afe87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:22.364492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ca356-3659-4426-a8db-f7a9e52afe87 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:22.366434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ca356-3659-4426-a8db-f7a9e52afe87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:22.368388 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ca356-3659-4426-a8db-f7a9e52afe87 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:22.385509 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1051/2096] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:21 2022] DELETE /identity/v3/users/7c56c23fc59646f7be726407dfccb83d => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:22.482506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1046/2097] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:22 2022] POST /identity/v3/users => generated 288 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:22.486040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211385ce-e068-406e-8ffa-7f4c32884975 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:22.487690 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-211385ce-e068-406e-8ffa-7f4c32884975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:22.582534 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211385ce-e068-406e-8ffa-7f4c32884975 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:22.599995 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec443a-d7d1-40e7-bb44-8fb51a4b118e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:22.603355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec443a-d7d1-40e7-bb44-8fb51a4b118e admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:22.603355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec443a-d7d1-40e7-bb44-8fb51a4b118e admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:22.603355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec443a-d7d1-40e7-bb44-8fb51a4b118e admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:22.603355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec443a-d7d1-40e7-bb44-8fb51a4b118e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:22.609604 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec443a-d7d1-40e7-bb44-8fb51a4b118e admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:22.637311 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1052/2098] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:18:22.639520 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ca356-3659-4426-a8db-f7a9e52afe87 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:22.667380 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ca356-3659-4426-a8db-f7a9e52afe87 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:22.679894 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf856b4-5f7b-43f0-b8f2-41cd96d0f887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:22.681515 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf856b4-5f7b-43f0-b8f2-41cd96d0f887 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:22.682061 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf856b4-5f7b-43f0-b8f2-41cd96d0f887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:22.682587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf856b4-5f7b-43f0-b8f2-41cd96d0f887 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:22.684505 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf856b4-5f7b-43f0-b8f2-41cd96d0f887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:22.688308 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf856b4-5f7b-43f0-b8f2-41cd96d0f887 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:22.713382 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1047/2099] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:22.717321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec443a-d7d1-40e7-bb44-8fb51a4b118e admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:22.736185 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec443a-d7d1-40e7-bb44-8fb51a4b118e admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:22.757233 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c13e4-259f-45ff-b1ec-709ddcde3ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:22.762393 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c13e4-259f-45ff-b1ec-709ddcde3ae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:22.762794 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c13e4-259f-45ff-b1ec-709ddcde3ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:22.765069 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c13e4-259f-45ff-b1ec-709ddcde3ae8 admin admin] PATH_INFO: `/v3/projects/332e2b023d4d42508a455845d0f6e52f/users/4c3123bcc38749058b7e2e697d827c8a/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:22.824528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c13e4-259f-45ff-b1ec-709ddcde3ae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=332e2b023d4d42508a455845d0f6e52f, user_id=4c3123bcc38749058b7e2e697d827c8a, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:22.836619 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c13e4-259f-45ff-b1ec-709ddcde3ae8 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:23.128742 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1053/2100] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:22 2022] PUT /identity/v3/projects/332e2b023d4d42508a455845d0f6e52f/users/4c3123bcc38749058b7e2e697d827c8a/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:23.140828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a448e70-b523-42c3-aeb5-961a57b3313f None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:23.140828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a448e70-b523-42c3-aeb5-961a57b3313f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:23.140828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a448e70-b523-42c3-aeb5-961a57b3313f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:23.211639 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-1a448e70-b523-42c3-aeb5-961a57b3313f None None] MFA Rules not processed for user `4c3123bcc38749058b7e2e697d827c8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:23.213669 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1a448e70-b523-42c3-aeb5-961a57b3313f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:23.296503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1048/2101] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:23.308166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c13e4-259f-45ff-b1ec-709ddcde3ae8 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:23.310193 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-996c13e4-259f-45ff-b1ec-709ddcde3ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:23.368985 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a448e70-b523-42c3-aeb5-961a57b3313f None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:23.372161 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1a448e70-b523-42c3-aeb5-961a57b3313f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:23.416505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c13e4-259f-45ff-b1ec-709ddcde3ae8 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:23.433650 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62227af2-03aa-41b2-997a-b8df012d65a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:23.435017 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62227af2-03aa-41b2-997a-b8df012d65a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:23.435460 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62227af2-03aa-41b2-997a-b8df012d65a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:23.436321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62227af2-03aa-41b2-997a-b8df012d65a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:23.450001 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62227af2-03aa-41b2-997a-b8df012d65a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:23.454898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62227af2-03aa-41b2-997a-b8df012d65a6 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:23.485618 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1054/2102] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:23.505001 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a448e70-b523-42c3-aeb5-961a57b3313f None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:23.509988 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ec08b-5866-408f-b2f0-0533587ec059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:23.511789 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ec08b-5866-408f-b2f0-0533587ec059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:23.511789 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ec08b-5866-408f-b2f0-0533587ec059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:23.511789 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ec08b-5866-408f-b2f0-0533587ec059 admin admin] PATH_INFO: `/v3/projects/cd8ea61cf687437e96faff4638cd3a48` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:23.512805 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ec08b-5866-408f-b2f0-0533587ec059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd8ea61cf687437e96faff4638cd3a48)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:23.514864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ec08b-5866-408f-b2f0-0533587ec059 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:23.640324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3f9ec08b-5866-408f-b2f0-0533587ec059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd8ea61cf687437e96faff4638cd3a48', 'request_id': 'req-3f9ec08b-5866-408f-b2f0-0533587ec059'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:24.198103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3f9ec08b-5866-408f-b2f0-0533587ec059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd8ea61cf687437e96faff4638cd3a48', 'request_id': 'req-3f9ec08b-5866-408f-b2f0-0533587ec059'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:24.206968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3f9ec08b-5866-408f-b2f0-0533587ec059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd8ea61cf687437e96faff4638cd3a48', 'request_id': 'req-3f9ec08b-5866-408f-b2f0-0533587ec059'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:24.222455 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3f9ec08b-5866-408f-b2f0-0533587ec059 admin admin] The token cache is being invalidate because project cd8ea61cf687437e96faff4638cd3a48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:24.222874 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3f9ec08b-5866-408f-b2f0-0533587ec059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:24.223725 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3f9ec08b-5866-408f-b2f0-0533587ec059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:24.226013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1049/2103] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:23 2022] DELETE /identity/v3/projects/cd8ea61cf687437e96faff4638cd3a48 => generated 0 bytes in 858 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:24.230951 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2af61-a651-4801-98bd-08d1763d728e None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:24.231465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2af61-a651-4801-98bd-08d1763d728e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:24.231653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2af61-a651-4801-98bd-08d1763d728e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:24.261904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-2ef2af61-a651-4801-98bd-08d1763d728e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:24.263374 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-2ef2af61-a651-4801-98bd-08d1763d728e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:24.348827 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1055/2104] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:24.365402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec8799-c4fd-41c5-b7e3-de4814ed94e8 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:24.365402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec8799-c4fd-41c5-b7e3-de4814ed94e8 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:24.365402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec8799-c4fd-41c5-b7e3-de4814ed94e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:24.389943 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-dbec8799-c4fd-41c5-b7e3-de4814ed94e8 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:24.391317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-dbec8799-c4fd-41c5-b7e3-de4814ed94e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:24.408607 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1050/2105] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:24.410313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2af61-a651-4801-98bd-08d1763d728e None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:24.422242 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2af61-a651-4801-98bd-08d1763d728e None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:24.437317 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454483c-1207-4895-9a36-4fa8906621ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:24.438476 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454483c-1207-4895-9a36-4fa8906621ec admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:24.438887 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454483c-1207-4895-9a36-4fa8906621ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:24.439278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454483c-1207-4895-9a36-4fa8906621ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:24.440214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454483c-1207-4895-9a36-4fa8906621ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:18:24.441324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454483c-1207-4895-9a36-4fa8906621ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:24.443798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454483c-1207-4895-9a36-4fa8906621ec admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:24.475844 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1056/2106] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 20:18:24.479413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec8799-c4fd-41c5-b7e3-de4814ed94e8 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:24.522743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec8799-c4fd-41c5-b7e3-de4814ed94e8 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:24.543618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a149fe-c64e-4311-aaf0-8e908d7ec08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:24.544626 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a149fe-c64e-4311-aaf0-8e908d7ec08a admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:24.545201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a149fe-c64e-4311-aaf0-8e908d7ec08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:24.547153 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a149fe-c64e-4311-aaf0-8e908d7ec08a admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:24.548010 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a149fe-c64e-4311-aaf0-8e908d7ec08a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:24.550031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a149fe-c64e-4311-aaf0-8e908d7ec08a admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:24.693775 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1051/2107] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:24 2022] POST /identity/v3/projects => generated 387 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:24.694620 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454483c-1207-4895-9a36-4fa8906621ec admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:24.757295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454483c-1207-4895-9a36-4fa8906621ec admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:24.863712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed5aac-7253-4102-9518-b62861d4bdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:24.864526 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed5aac-7253-4102-9518-b62861d4bdae admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:24.864951 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed5aac-7253-4102-9518-b62861d4bdae admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:24.865456 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed5aac-7253-4102-9518-b62861d4bdae admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:24.867100 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed5aac-7253-4102-9518-b62861d4bdae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:24.869023 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed5aac-7253-4102-9518-b62861d4bdae admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:25.057859 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1057/2108] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:24 2022] POST /identity/v3/users => generated 294 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:18:25.059469 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a149fe-c64e-4311-aaf0-8e908d7ec08a admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:25.091315 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a149fe-c64e-4311-aaf0-8e908d7ec08a admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:25.096652 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8135e67-c53d-4e94-a84c-60cc62811618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:25.097337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8135e67-c53d-4e94-a84c-60cc62811618 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:25.097622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8135e67-c53d-4e94-a84c-60cc62811618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:25.097899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8135e67-c53d-4e94-a84c-60cc62811618 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:25.098790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8135e67-c53d-4e94-a84c-60cc62811618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:25.101003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8135e67-c53d-4e94-a84c-60cc62811618 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:25.113168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1052/2109] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:25.117025 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed5aac-7253-4102-9518-b62861d4bdae admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:25.127263 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed5aac-7253-4102-9518-b62861d4bdae admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:25.139657 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd33534-d143-4145-8f4e-03fa2b087254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:25.139657 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd33534-d143-4145-8f4e-03fa2b087254 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:25.139657 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd33534-d143-4145-8f4e-03fa2b087254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:25.139657 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd33534-d143-4145-8f4e-03fa2b087254 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:25.140037 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd33534-d143-4145-8f4e-03fa2b087254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:25.140037 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd33534-d143-4145-8f4e-03fa2b087254 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:25.149945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1058/2110] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:25.154234 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8135e67-c53d-4e94-a84c-60cc62811618 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:25.166217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8135e67-c53d-4e94-a84c-60cc62811618 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:25.172907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e14faf-75ff-413c-a0a2-9832aa2c368d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:25.173799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e14faf-75ff-413c-a0a2-9832aa2c368d admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:25.174119 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e14faf-75ff-413c-a0a2-9832aa2c368d admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:25.174403 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e14faf-75ff-413c-a0a2-9832aa2c368d admin admin] PATH_INFO: `/v3/projects/21bbfd36c3cd40358bc31785f4be4dcb/users/db00238b222d43fba89c2e72542661c9/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:25.191372 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e14faf-75ff-413c-a0a2-9832aa2c368d admin admin] RBAC: Authorizing `identity:create_grant(project_id=21bbfd36c3cd40358bc31785f4be4dcb, user_id=db00238b222d43fba89c2e72542661c9, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:25.194601 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e14faf-75ff-413c-a0a2-9832aa2c368d admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:25.286066 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1053/2111] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:25 2022] PUT /identity/v3/projects/21bbfd36c3cd40358bc31785f4be4dcb/users/db00238b222d43fba89c2e72542661c9/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:25.289498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd33534-d143-4145-8f4e-03fa2b087254 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:25.291644 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-cfd33534-d143-4145-8f4e-03fa2b087254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:25.297391 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40021d6f-388e-4d9e-be02-6ce606645b4e None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:25.297744 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40021d6f-388e-4d9e-be02-6ce606645b4e None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:25.298110 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40021d6f-388e-4d9e-be02-6ce606645b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:25.305802 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd33534-d143-4145-8f4e-03fa2b087254 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:25.329402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-40021d6f-388e-4d9e-be02-6ce606645b4e None None] MFA Rules not processed for user `db00238b222d43fba89c2e72542661c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:25.330752 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-40021d6f-388e-4d9e-be02-6ce606645b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:25.367432 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e29d64-f1a3-47cd-afeb-a9e0bae6087a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:25.368308 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e29d64-f1a3-47cd-afeb-a9e0bae6087a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:25.368591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e29d64-f1a3-47cd-afeb-a9e0bae6087a admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:25.368871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e29d64-f1a3-47cd-afeb-a9e0bae6087a admin admin] PATH_INFO: `/v3/users/13e7ca97b2434d218ce6101528965ef0` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:25.370193 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e29d64-f1a3-47cd-afeb-a9e0bae6087a admin admin] RBAC: Authorizing `identity:delete_user(user_id=13e7ca97b2434d218ce6101528965ef0)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:25.372175 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e29d64-f1a3-47cd-afeb-a9e0bae6087a admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:25.404899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1054/2112] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:18:25.409263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40021d6f-388e-4d9e-be02-6ce606645b4e None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:25.422227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40021d6f-388e-4d9e-be02-6ce606645b4e None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:25.425136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c96ed-2b29-4307-b885-6d8ef7065692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:25.425829 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c96ed-2b29-4307-b885-6d8ef7065692 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:25.426126 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c96ed-2b29-4307-b885-6d8ef7065692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:25.426400 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c96ed-2b29-4307-b885-6d8ef7065692 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:25.427088 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c96ed-2b29-4307-b885-6d8ef7065692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:25.429589 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c96ed-2b29-4307-b885-6d8ef7065692 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:25.539918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1055/2113] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:25 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:18:25.539918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b947f-692e-4ca1-9e88-d4e102a2b98e None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:25.539918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b947f-692e-4ca1-9e88-d4e102a2b98e None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:25.539918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b947f-692e-4ca1-9e88-d4e102a2b98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:25.556322 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-618b947f-692e-4ca1-9e88-d4e102a2b98e None None] MFA Rules not processed for user `6f5b649235d646fea59e7e61e7e6d8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:25.558432 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-618b947f-692e-4ca1-9e88-d4e102a2b98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:25.613157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1056/2114] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:25.614103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b947f-692e-4ca1-9e88-d4e102a2b98e None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:25.638894 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-d7e29d64-f1a3-47cd-afeb-a9e0bae6087a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13e7ca97b2434d218ce6101528965ef0'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:25.639680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-d7e29d64-f1a3-47cd-afeb-a9e0bae6087a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13e7ca97b2434d218ce6101528965ef0'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:25.640362 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-d7e29d64-f1a3-47cd-afeb-a9e0bae6087a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13e7ca97b2434d218ce6101528965ef0'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:25.654162 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-d7e29d64-f1a3-47cd-afeb-a9e0bae6087a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13e7ca97b2434d218ce6101528965ef0'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:25.661206 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-d7e29d64-f1a3-47cd-afeb-a9e0bae6087a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13e7ca97b2434d218ce6101528965ef0'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:25.681567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b947f-692e-4ca1-9e88-d4e102a2b98e None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:25.690524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbabbfb-1c3f-471c-ac44-d4a93fd7b98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:25.690524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbabbfb-1c3f-471c-ac44-d4a93fd7b98d admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:25.690524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbabbfb-1c3f-471c-ac44-d4a93fd7b98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:25.690524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbabbfb-1c3f-471c-ac44-d4a93fd7b98d admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:25.690973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbabbfb-1c3f-471c-ac44-d4a93fd7b98d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:25.691219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbabbfb-1c3f-471c-ac44-d4a93fd7b98d admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:25.743183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1059/2115] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:25 2022] DELETE /identity/v3/users/13e7ca97b2434d218ce6101528965ef0 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:25.747671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fb830-b253-46e6-8ad3-f15860260443 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:25.748165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fb830-b253-46e6-8ad3-f15860260443 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:25.748447 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fb830-b253-46e6-8ad3-f15860260443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:25.767711 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-d83fb830-b253-46e6-8ad3-f15860260443 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:25.768906 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-d83fb830-b253-46e6-8ad3-f15860260443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:25.841221 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1060/2116] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:25.841327 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1057/2117] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:25 2022] POST /identity/v3/users => generated 299 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:25.841424 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83fb830-b253-46e6-8ad3-f15860260443 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:25.841510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbabbfb-1c3f-471c-ac44-d4a93fd7b98d admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:25.841599 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-d83fb830-b253-46e6-8ad3-f15860260443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:25.841684 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-5fbabbfb-1c3f-471c-ac44-d4a93fd7b98d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:25.854859 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83fb830-b253-46e6-8ad3-f15860260443 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:25.864294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb9097-660f-464a-bfc3-43023d6b9ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:25.865623 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb9097-660f-464a-bfc3-43023d6b9ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:25.866189 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb9097-660f-464a-bfc3-43023d6b9ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:25.867210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb9097-660f-464a-bfc3-43023d6b9ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:25.868132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb9097-660f-464a-bfc3-43023d6b9ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:25.870199 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb9097-660f-464a-bfc3-43023d6b9ae9 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:25.892226 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1061/2118] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:25.893843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb9097-660f-464a-bfc3-43023d6b9ae9 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:25.905468 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb9097-660f-464a-bfc3-43023d6b9ae9 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:25.913293 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec2a4e-e21a-4be4-a180-be88d3e1313e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:25.915987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec2a4e-e21a-4be4-a180-be88d3e1313e admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:25.915987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec2a4e-e21a-4be4-a180-be88d3e1313e admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:25.915987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec2a4e-e21a-4be4-a180-be88d3e1313e admin admin] PATH_INFO: `/v3/projects/08e3d7ef9cb34e72b8c9685459706c93/users/8473d3991c6d4a88957d54a9d72c439c/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:25.938114 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec2a4e-e21a-4be4-a180-be88d3e1313e admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e3d7ef9cb34e72b8c9685459706c93, user_id=8473d3991c6d4a88957d54a9d72c439c, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:25.943203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec2a4e-e21a-4be4-a180-be88d3e1313e admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:25.957752 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbabbfb-1c3f-471c-ac44-d4a93fd7b98d admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:26.027480 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1062/2119] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:25 2022] PUT /identity/v3/projects/08e3d7ef9cb34e72b8c9685459706c93/users/8473d3991c6d4a88957d54a9d72c439c/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:26.032900 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec2a4e-e21a-4be4-a180-be88d3e1313e admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:26.034351 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b4723d-62e3-43f4-98a7-8c114d5a759c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:26.035278 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4723d-62e3-43f4-98a7-8c114d5a759c service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:26.035780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4723d-62e3-43f4-98a7-8c114d5a759c service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:26.036287 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4723d-62e3-43f4-98a7-8c114d5a759c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:26.056807 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b4723d-62e3-43f4-98a7-8c114d5a759c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:26.059952 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b4723d-62e3-43f4-98a7-8c114d5a759c service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:26.075006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1058/2120] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:26.128978 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec2a4e-e21a-4be4-a180-be88d3e1313e admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:26.134054 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7fb48-c39f-4ed5-bddc-bc6848e0c78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:26.134876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7fb48-c39f-4ed5-bddc-bc6848e0c78d admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:26.135263 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7fb48-c39f-4ed5-bddc-bc6848e0c78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:26.135640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7fb48-c39f-4ed5-bddc-bc6848e0c78d admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:26.136628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f7fb48-c39f-4ed5-bddc-bc6848e0c78d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:26.144395 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f7fb48-c39f-4ed5-bddc-bc6848e0c78d admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:26.158559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1063/2121] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:26.164579 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b4723d-62e3-43f4-98a7-8c114d5a759c service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:26.175329 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b4723d-62e3-43f4-98a7-8c114d5a759c service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:26.182047 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646fe4d-4c2f-458d-9d2d-6fd5483a7086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:26.183218 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646fe4d-4c2f-458d-9d2d-6fd5483a7086 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:26.183639 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646fe4d-4c2f-458d-9d2d-6fd5483a7086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:26.184012 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646fe4d-4c2f-458d-9d2d-6fd5483a7086 admin admin] PATH_INFO: `/v3/projects/08e3d7ef9cb34e72b8c9685459706c93/users/8473d3991c6d4a88957d54a9d72c439c/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:26.186675 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646fe4d-4c2f-458d-9d2d-6fd5483a7086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e3d7ef9cb34e72b8c9685459706c93, user_id=8473d3991c6d4a88957d54a9d72c439c, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:26.190696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646fe4d-4c2f-458d-9d2d-6fd5483a7086 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:26.215860 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1059/2122] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:26 2022] PUT /identity/v3/projects/08e3d7ef9cb34e72b8c9685459706c93/users/8473d3991c6d4a88957d54a9d72c439c/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:26.217636 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbecd6-e6df-4bc2-8899-ab48a44a925b None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:26.217929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbecd6-e6df-4bc2-8899-ab48a44a925b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:26.218239 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbecd6-e6df-4bc2-8899-ab48a44a925b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:26.253014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-38cbecd6-e6df-4bc2-8899-ab48a44a925b None None] MFA Rules not processed for user `8473d3991c6d4a88957d54a9d72c439c`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:26.257210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-38cbecd6-e6df-4bc2-8899-ab48a44a925b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:26.322201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1064/2123] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:26.333102 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646fe4d-4c2f-458d-9d2d-6fd5483a7086 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:26.335313 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-9646fe4d-4c2f-458d-9d2d-6fd5483a7086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:26.422531 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646fe4d-4c2f-458d-9d2d-6fd5483a7086 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:26.435252 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed56a1-5234-4b22-8658-dae35a1a9864 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:26.436859 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed56a1-5234-4b22-8658-dae35a1a9864 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:26.437246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed56a1-5234-4b22-8658-dae35a1a9864 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:26.437631 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed56a1-5234-4b22-8658-dae35a1a9864 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:26.448981 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed56a1-5234-4b22-8658-dae35a1a9864 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:26.450749 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed56a1-5234-4b22-8658-dae35a1a9864 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:26.464241 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1060/2124] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:26.677629 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbecd6-e6df-4bc2-8899-ab48a44a925b None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:26.705136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed56a1-5234-4b22-8658-dae35a1a9864 service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:26.709860 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-4fed56a1-5234-4b22-8658-dae35a1a9864 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:26.772518 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbecd6-e6df-4bc2-8899-ab48a44a925b None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:26.777225 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa6372-503b-41ea-a346-49e9cd84ce1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:26.778124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa6372-503b-41ea-a346-49e9cd84ce1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:26.778437 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa6372-503b-41ea-a346-49e9cd84ce1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:26.778715 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa6372-503b-41ea-a346-49e9cd84ce1a admin admin] PATH_INFO: `/v3/users/db00238b222d43fba89c2e72542661c9` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:26.779917 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa6372-503b-41ea-a346-49e9cd84ce1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=db00238b222d43fba89c2e72542661c9)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:26.782137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa6372-503b-41ea-a346-49e9cd84ce1a admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:26.794905 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed56a1-5234-4b22-8658-dae35a1a9864 service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:26.799533 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76406c2-a5da-4fc5-95d6-368781210618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:26.800420 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76406c2-a5da-4fc5-95d6-368781210618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:26.800706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76406c2-a5da-4fc5-95d6-368781210618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:26.800983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76406c2-a5da-4fc5-95d6-368781210618 admin admin] PATH_INFO: `/v3/projects/448215ba651142c9a5c2a03ca1951701` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:26.802219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76406c2-a5da-4fc5-95d6-368781210618 admin admin] RBAC: Authorizing `identity:delete_project(project_id=448215ba651142c9a5c2a03ca1951701)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:26.804846 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76406c2-a5da-4fc5-95d6-368781210618 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:26.999215 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f76406c2-a5da-4fc5-95d6-368781210618 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '448215ba651142c9a5c2a03ca1951701', 'request_id': 'req-f76406c2-a5da-4fc5-95d6-368781210618'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:27.012747 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a0fa6372-503b-41ea-a346-49e9cd84ce1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db00238b222d43fba89c2e72542661c9'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:27.013522 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a0fa6372-503b-41ea-a346-49e9cd84ce1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db00238b222d43fba89c2e72542661c9'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:27.014177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a0fa6372-503b-41ea-a346-49e9cd84ce1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db00238b222d43fba89c2e72542661c9'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:27.027212 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a0fa6372-503b-41ea-a346-49e9cd84ce1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db00238b222d43fba89c2e72542661c9'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:27.033874 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a0fa6372-503b-41ea-a346-49e9cd84ce1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db00238b222d43fba89c2e72542661c9'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:27.071939 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f76406c2-a5da-4fc5-95d6-368781210618 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '448215ba651142c9a5c2a03ca1951701', 'request_id': 'req-f76406c2-a5da-4fc5-95d6-368781210618'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:27.078331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f76406c2-a5da-4fc5-95d6-368781210618 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '448215ba651142c9a5c2a03ca1951701', 'request_id': 'req-f76406c2-a5da-4fc5-95d6-368781210618'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:27.090149 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f76406c2-a5da-4fc5-95d6-368781210618 admin admin] The token cache is being invalidate because project 448215ba651142c9a5c2a03ca1951701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:27.090499 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f76406c2-a5da-4fc5-95d6-368781210618 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:27.091159 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f76406c2-a5da-4fc5-95d6-368781210618 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:27.092690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1061/2125] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:26 2022] DELETE /identity/v3/projects/448215ba651142c9a5c2a03ca1951701 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:27.096582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76406c2-a5da-4fc5-95d6-368781210618 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:27.098292 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f76406c2-a5da-4fc5-95d6-368781210618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:27.162371 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76406c2-a5da-4fc5-95d6-368781210618 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:27.167291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea73828-510f-4f09-9759-0c298d7243f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:27.168147 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea73828-510f-4f09-9759-0c298d7243f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:27.168428 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea73828-510f-4f09-9759-0c298d7243f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:27.168699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea73828-510f-4f09-9759-0c298d7243f4 admin admin] PATH_INFO: `/v3/users/7e23ecfd75c148bf948940d7fffe51c6` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:27.169522 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1065/2126] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:26 2022] DELETE /identity/v3/users/db00238b222d43fba89c2e72542661c9 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:27.169898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea73828-510f-4f09-9759-0c298d7243f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e23ecfd75c148bf948940d7fffe51c6)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:27.171842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea73828-510f-4f09-9759-0c298d7243f4 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:27.180739 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c0f71-3aae-4878-ae94-57a3e1d0fd05 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:27.180739 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c0f71-3aae-4878-ae94-57a3e1d0fd05 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:27.180739 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c0f71-3aae-4878-ae94-57a3e1d0fd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:27.197712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-4e2c0f71-3aae-4878-ae94-57a3e1d0fd05 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:27.199149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4e2c0f71-3aae-4878-ae94-57a3e1d0fd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:27.266940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1066/2127] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:18:27.273604 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c0f71-3aae-4878-ae94-57a3e1d0fd05 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:27.278793 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4e2c0f71-3aae-4878-ae94-57a3e1d0fd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:27.348734 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c0f71-3aae-4878-ae94-57a3e1d0fd05 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:27.356451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96c859-ef9c-4c38-9a8c-b4ff3be2b21c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:27.357004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96c859-ef9c-4c38-9a8c-b4ff3be2b21c service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:27.357288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96c859-ef9c-4c38-9a8c-b4ff3be2b21c service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:27.357563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96c859-ef9c-4c38-9a8c-b4ff3be2b21c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:27.367575 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96c859-ef9c-4c38-9a8c-b4ff3be2b21c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:27.368996 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96c859-ef9c-4c38-9a8c-b4ff3be2b21c service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:27.378752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1067/2128] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:18:27.404431 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4ea73828-510f-4f09-9759-0c298d7243f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e23ecfd75c148bf948940d7fffe51c6'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:27.405511 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4ea73828-510f-4f09-9759-0c298d7243f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e23ecfd75c148bf948940d7fffe51c6'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:27.427313 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4ea73828-510f-4f09-9759-0c298d7243f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e23ecfd75c148bf948940d7fffe51c6'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:27.427313 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4ea73828-510f-4f09-9759-0c298d7243f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e23ecfd75c148bf948940d7fffe51c6'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:27.438566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4ea73828-510f-4f09-9759-0c298d7243f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e23ecfd75c148bf948940d7fffe51c6'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:27.534820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1062/2129] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:27 2022] DELETE /identity/v3/users/7e23ecfd75c148bf948940d7fffe51c6 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:27.913486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96c859-ef9c-4c38-9a8c-b4ff3be2b21c service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:27.914983 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-1f96c859-ef9c-4c38-9a8c-b4ff3be2b21c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:27.933391 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96c859-ef9c-4c38-9a8c-b4ff3be2b21c service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:27.992963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7c0d1-6f7b-4115-b470-bbdda4403dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:27.993821 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7c0d1-6f7b-4115-b470-bbdda4403dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:27.994136 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7c0d1-6f7b-4115-b470-bbdda4403dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:27.994419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7c0d1-6f7b-4115-b470-bbdda4403dcb admin admin] PATH_INFO: `/v3/projects/21bbfd36c3cd40358bc31785f4be4dcb` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:27.995628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7c0d1-6f7b-4115-b470-bbdda4403dcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=21bbfd36c3cd40358bc31785f4be4dcb)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:27.997532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7c0d1-6f7b-4115-b470-bbdda4403dcb admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:28.062196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea73828-510f-4f09-9759-0c298d7243f4 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:28.063665 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-4ea73828-510f-4f09-9759-0c298d7243f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:28.087832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea73828-510f-4f09-9759-0c298d7243f4 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:28.101124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-0bd7c0d1-6f7b-4115-b470-bbdda4403dcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21bbfd36c3cd40358bc31785f4be4dcb', 'request_id': 'req-0bd7c0d1-6f7b-4115-b470-bbdda4403dcb'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.151528 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d88c22-74b2-4263-87bf-cf841c04c085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:28.152384 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d88c22-74b2-4263-87bf-cf841c04c085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:28.152665 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d88c22-74b2-4263-87bf-cf841c04c085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:28.152981 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d88c22-74b2-4263-87bf-cf841c04c085 admin admin] PATH_INFO: `/v3/projects/785547e501014f8ba5db26397d031c8c` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:28.154205 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d88c22-74b2-4263-87bf-cf841c04c085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=785547e501014f8ba5db26397d031c8c)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:28.156140 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d88c22-74b2-4263-87bf-cf841c04c085 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:28.177403 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-0bd7c0d1-6f7b-4115-b470-bbdda4403dcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21bbfd36c3cd40358bc31785f4be4dcb', 'request_id': 'req-0bd7c0d1-6f7b-4115-b470-bbdda4403dcb'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.183749 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-0bd7c0d1-6f7b-4115-b470-bbdda4403dcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21bbfd36c3cd40358bc31785f4be4dcb', 'request_id': 'req-0bd7c0d1-6f7b-4115-b470-bbdda4403dcb'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.195666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-0bd7c0d1-6f7b-4115-b470-bbdda4403dcb admin admin] The token cache is being invalidate because project 21bbfd36c3cd40358bc31785f4be4dcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:28.195956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-0bd7c0d1-6f7b-4115-b470-bbdda4403dcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.196737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-0bd7c0d1-6f7b-4115-b470-bbdda4403dcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.198371 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1068/2130] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:27 2022] DELETE /identity/v3/projects/21bbfd36c3cd40358bc31785f4be4dcb => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:28.202046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7c0d1-6f7b-4115-b470-bbdda4403dcb admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:28.203344 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-0bd7c0d1-6f7b-4115-b470-bbdda4403dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:28.217398 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7c0d1-6f7b-4115-b470-bbdda4403dcb admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:28.222140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc3f2b-b823-49fb-9bfd-926ec7826f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:28.223076 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc3f2b-b823-49fb-9bfd-926ec7826f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:28.223365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc3f2b-b823-49fb-9bfd-926ec7826f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:28.223645 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc3f2b-b823-49fb-9bfd-926ec7826f02 admin admin] PATH_INFO: `/v3/users/8473d3991c6d4a88957d54a9d72c439c` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:28.224904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc3f2b-b823-49fb-9bfd-926ec7826f02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8473d3991c6d4a88957d54a9d72c439c)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:28.226889 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc3f2b-b823-49fb-9bfd-926ec7826f02 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:28.295053 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-14d88c22-74b2-4263-87bf-cf841c04c085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '785547e501014f8ba5db26397d031c8c', 'request_id': 'req-14d88c22-74b2-4263-87bf-cf841c04c085'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.406050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-14d88c22-74b2-4263-87bf-cf841c04c085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '785547e501014f8ba5db26397d031c8c', 'request_id': 'req-14d88c22-74b2-4263-87bf-cf841c04c085'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.414916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-14d88c22-74b2-4263-87bf-cf841c04c085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '785547e501014f8ba5db26397d031c8c', 'request_id': 'req-14d88c22-74b2-4263-87bf-cf841c04c085'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.436424 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-14d88c22-74b2-4263-87bf-cf841c04c085 admin admin] The token cache is being invalidate because project 785547e501014f8ba5db26397d031c8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:28.436594 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-14d88c22-74b2-4263-87bf-cf841c04c085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.437441 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-14d88c22-74b2-4263-87bf-cf841c04c085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.449286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1063/2131] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:28 2022] DELETE /identity/v3/projects/785547e501014f8ba5db26397d031c8c => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:28.450468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d88c22-74b2-4263-87bf-cf841c04c085 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:28.452071 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-14d88c22-74b2-4263-87bf-cf841c04c085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:28.486329 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a5dc3f2b-b823-49fb-9bfd-926ec7826f02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8473d3991c6d4a88957d54a9d72c439c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.487056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a5dc3f2b-b823-49fb-9bfd-926ec7826f02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8473d3991c6d4a88957d54a9d72c439c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.487666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a5dc3f2b-b823-49fb-9bfd-926ec7826f02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8473d3991c6d4a88957d54a9d72c439c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.501110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a5dc3f2b-b823-49fb-9bfd-926ec7826f02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8473d3991c6d4a88957d54a9d72c439c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.508724 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a5dc3f2b-b823-49fb-9bfd-926ec7826f02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8473d3991c6d4a88957d54a9d72c439c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.523673 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d88c22-74b2-4263-87bf-cf841c04c085 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:28.528567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347587e-eadd-4420-ba97-f544846501f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:28.529453 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347587e-eadd-4420-ba97-f544846501f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:28.529725 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347587e-eadd-4420-ba97-f544846501f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:28.530162 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347587e-eadd-4420-ba97-f544846501f0 admin admin] PATH_INFO: `/v3/users/40ee22e521f14608acc89b98ba45002e` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:28.531377 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347587e-eadd-4420-ba97-f544846501f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40ee22e521f14608acc89b98ba45002e)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:28.533294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347587e-eadd-4420-ba97-f544846501f0 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:28.595894 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1069/2132] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:28 2022] DELETE /identity/v3/users/8473d3991c6d4a88957d54a9d72c439c => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:28.785159 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4347587e-eadd-4420-ba97-f544846501f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40ee22e521f14608acc89b98ba45002e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.785804 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4347587e-eadd-4420-ba97-f544846501f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40ee22e521f14608acc89b98ba45002e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.798108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4347587e-eadd-4420-ba97-f544846501f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40ee22e521f14608acc89b98ba45002e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.798786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4347587e-eadd-4420-ba97-f544846501f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40ee22e521f14608acc89b98ba45002e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.805792 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4347587e-eadd-4420-ba97-f544846501f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40ee22e521f14608acc89b98ba45002e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:28.894015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1064/2133] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:28 2022] DELETE /identity/v3/users/40ee22e521f14608acc89b98ba45002e => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:29.088258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc3f2b-b823-49fb-9bfd-926ec7826f02 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:29.089839 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a5dc3f2b-b823-49fb-9bfd-926ec7826f02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:29.163905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc3f2b-b823-49fb-9bfd-926ec7826f02 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:29.269566 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6fd72-b4f8-4534-8d84-f10044877f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:29.271096 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6fd72-b4f8-4534-8d84-f10044877f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:29.272488 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6fd72-b4f8-4534-8d84-f10044877f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:29.273024 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6fd72-b4f8-4534-8d84-f10044877f6f admin admin] PATH_INFO: `/v3/projects/08e3d7ef9cb34e72b8c9685459706c93` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:29.274551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6fd72-b4f8-4534-8d84-f10044877f6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=08e3d7ef9cb34e72b8c9685459706c93)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:29.277082 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6fd72-b4f8-4534-8d84-f10044877f6f admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:29.409394 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-0dd6fd72-b4f8-4534-8d84-f10044877f6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08e3d7ef9cb34e72b8c9685459706c93', 'request_id': 'req-0dd6fd72-b4f8-4534-8d84-f10044877f6f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:29.515003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-0dd6fd72-b4f8-4534-8d84-f10044877f6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08e3d7ef9cb34e72b8c9685459706c93', 'request_id': 'req-0dd6fd72-b4f8-4534-8d84-f10044877f6f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:29.524819 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-0dd6fd72-b4f8-4534-8d84-f10044877f6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08e3d7ef9cb34e72b8c9685459706c93', 'request_id': 'req-0dd6fd72-b4f8-4534-8d84-f10044877f6f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:29.555315 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-0dd6fd72-b4f8-4534-8d84-f10044877f6f admin admin] The token cache is being invalidate because project 08e3d7ef9cb34e72b8c9685459706c93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:29.555315 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-0dd6fd72-b4f8-4534-8d84-f10044877f6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:29.555315 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-0dd6fd72-b4f8-4534-8d84-f10044877f6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:29.560000 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1070/2134] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:29 2022] DELETE /identity/v3/projects/08e3d7ef9cb34e72b8c9685459706c93 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:29.560718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347587e-eadd-4420-ba97-f544846501f0 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:29.562489 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffd8f2-ea64-43c3-b66a-e7b130e3aae2 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:29.562725 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-4347587e-eadd-4420-ba97-f544846501f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:29.563104 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffd8f2-ea64-43c3-b66a-e7b130e3aae2 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:29.563539 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffd8f2-ea64-43c3-b66a-e7b130e3aae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:29.620022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347587e-eadd-4420-ba97-f544846501f0 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:29.629270 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-65ffd8f2-ea64-43c3-b66a-e7b130e3aae2 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:29.630693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-65ffd8f2-ea64-43c3-b66a-e7b130e3aae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:29.706465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1071/2135] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:29.711630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75c742-b62f-4368-8db8-205b3e7b496c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:29.711925 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75c742-b62f-4368-8db8-205b3e7b496c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:29.712208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75c742-b62f-4368-8db8-205b3e7b496c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:29.729048 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d87f0-dffc-4b7d-8c72-91a99164d377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:29.729832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d87f0-dffc-4b7d-8c72-91a99164d377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:29.730136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d87f0-dffc-4b7d-8c72-91a99164d377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:29.730702 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d87f0-dffc-4b7d-8c72-91a99164d377 admin admin] PATH_INFO: `/v3/projects/b61e26c29698425a99575213eaf1bc46` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:29.732326 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d87f0-dffc-4b7d-8c72-91a99164d377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b61e26c29698425a99575213eaf1bc46)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:29.734415 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d87f0-dffc-4b7d-8c72-91a99164d377 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:29.777075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-4c75c742-b62f-4368-8db8-205b3e7b496c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:29.777075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4c75c742-b62f-4368-8db8-205b3e7b496c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:29.783929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1072/2136] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:29.788677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75c742-b62f-4368-8db8-205b3e7b496c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:29.799814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75c742-b62f-4368-8db8-205b3e7b496c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:29.805677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812a045-5cbe-4ff3-ad87-5c572c3b1493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:29.806555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812a045-5cbe-4ff3-ad87-5c572c3b1493 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:29.806845 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812a045-5cbe-4ff3-ad87-5c572c3b1493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:29.807113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812a045-5cbe-4ff3-ad87-5c572c3b1493 admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:29.808251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812a045-5cbe-4ff3-ad87-5c572c3b1493 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:18:29.808569 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812a045-5cbe-4ff3-ad87-5c572c3b1493 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:29.810621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812a045-5cbe-4ff3-ad87-5c572c3b1493 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:29.832853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1073/2137] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:18:29.834046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fdf0d-af5a-43f7-b2e8-156052f9b7bc None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:29.835016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fdf0d-af5a-43f7-b2e8-156052f9b7bc None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:29.835272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fdf0d-af5a-43f7-b2e8-156052f9b7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:29.853797 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-f29fdf0d-af5a-43f7-b2e8-156052f9b7bc None None] MFA Rules not processed for user `7ec2605fd1b84e0ea134b943ed33a605`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:29.855131 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-f29fdf0d-af5a-43f7-b2e8-156052f9b7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:29.955338 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1074/2138] 10.222.0.65 () {58 vars in 1009 bytes} [Tue May 10 20:18:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:29.956200 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fdf0d-af5a-43f7-b2e8-156052f9b7bc None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:29.967174 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-482d87f0-dffc-4b7d-8c72-91a99164d377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b61e26c29698425a99575213eaf1bc46', 'request_id': 'req-482d87f0-dffc-4b7d-8c72-91a99164d377'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:29.972295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fdf0d-af5a-43f7-b2e8-156052f9b7bc None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:30.049149 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-482d87f0-dffc-4b7d-8c72-91a99164d377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b61e26c29698425a99575213eaf1bc46', 'request_id': 'req-482d87f0-dffc-4b7d-8c72-91a99164d377'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:30.049440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d405f7-b700-4f39-8b3b-9301fa6a470f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:30.050329 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d405f7-b700-4f39-8b3b-9301fa6a470f admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:30.056753 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d405f7-b700-4f39-8b3b-9301fa6a470f admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:30.056753 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d405f7-b700-4f39-8b3b-9301fa6a470f admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:30.056753 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d405f7-b700-4f39-8b3b-9301fa6a470f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:30.057288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-482d87f0-dffc-4b7d-8c72-91a99164d377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b61e26c29698425a99575213eaf1bc46', 'request_id': 'req-482d87f0-dffc-4b7d-8c72-91a99164d377'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:30.057365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d405f7-b700-4f39-8b3b-9301fa6a470f admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:30.069739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-482d87f0-dffc-4b7d-8c72-91a99164d377 admin admin] The token cache is being invalidate because project b61e26c29698425a99575213eaf1bc46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:30.070123 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-482d87f0-dffc-4b7d-8c72-91a99164d377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:30.070692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-482d87f0-dffc-4b7d-8c72-91a99164d377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:30.072473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1065/2139] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:29 2022] DELETE /identity/v3/projects/b61e26c29698425a99575213eaf1bc46 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:30.073603 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d87f0-dffc-4b7d-8c72-91a99164d377 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:30.074608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-482d87f0-dffc-4b7d-8c72-91a99164d377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:30.152797 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1075/2140] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:29 2022] POST /identity/v3/projects => generated 383 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:30.155091 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ed9df-c43a-431e-9e76-c5a7a028feb8 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:30.155611 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ed9df-c43a-431e-9e76-c5a7a028feb8 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:30.156396 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ed9df-c43a-431e-9e76-c5a7a028feb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:30.176547 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d87f0-dffc-4b7d-8c72-91a99164d377 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:30.200841 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-a94ed9df-c43a-431e-9e76-c5a7a028feb8 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:30.202112 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a94ed9df-c43a-431e-9e76-c5a7a028feb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:30.318258 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d8a9f-96bc-434f-8443-1067c46220ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d10b258a53484da22950e2b1db2e41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:30.319725 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d8a9f-96bc-434f-8443-1067c46220ea service swift] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:30.320291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d8a9f-96bc-434f-8443-1067c46220ea service swift] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:30.320291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d8a9f-96bc-434f-8443-1067c46220ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:30.322227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0e2d8a9f-96bc-434f-8443-1067c46220ea service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:30.339158 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1076/2141] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:30.340619 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ed9df-c43a-431e-9e76-c5a7a028feb8 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:30.348064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a94ed9df-c43a-431e-9e76-c5a7a028feb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:30.366642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ed9df-c43a-431e-9e76-c5a7a028feb8 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:30.374122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f08fec-1e3c-41ee-904f-d020293efb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:30.375315 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f08fec-1e3c-41ee-904f-d020293efb25 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:30.375801 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f08fec-1e3c-41ee-904f-d020293efb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:30.376225 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f08fec-1e3c-41ee-904f-d020293efb25 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:30.377078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f08fec-1e3c-41ee-904f-d020293efb25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:30.380365 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f08fec-1e3c-41ee-904f-d020293efb25 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:30.467244 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d8a9f-96bc-434f-8443-1067c46220ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:30.477792 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1077/2142] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:30 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:30.479234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d82ba4-d00d-4020-b13f-8f6bade2b56c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:30.479454 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d82ba4-d00d-4020-b13f-8f6bade2b56c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:30.483412 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d82ba4-d00d-4020-b13f-8f6bade2b56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:30.486791 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d8a9f-96bc-434f-8443-1067c46220ea service swift] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:30.508982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1066/2143] 10.222.0.65 () {60 vars in 1384 bytes} [Tue May 10 20:18:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 20:18:30.511582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d8a9f-96bc-434f-8443-1067c46220ea service swift] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:30.530617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-13d82ba4-d00d-4020-b13f-8f6bade2b56c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:30.532052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-13d82ba4-d00d-4020-b13f-8f6bade2b56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:30.537279 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d8a9f-96bc-434f-8443-1067c46220ea service swift] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:30.543132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1078/2144] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:30.547189 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d82ba4-d00d-4020-b13f-8f6bade2b56c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:30.559974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b0a20-406b-4a6a-ad43-0113d01b1fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:30.563387 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b0a20-406b-4a6a-ad43-0113d01b1fca admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:30.563387 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b0a20-406b-4a6a-ad43-0113d01b1fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:30.563387 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b0a20-406b-4a6a-ad43-0113d01b1fca admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:30.564120 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d82ba4-d00d-4020-b13f-8f6bade2b56c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:30.564212 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b0a20-406b-4a6a-ad43-0113d01b1fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:30.564454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b0a20-406b-4a6a-ad43-0113d01b1fca admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:30.569039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57267e-88d3-4def-a78b-2090e952b8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:30.569837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57267e-88d3-4def-a78b-2090e952b8ad admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:30.570240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57267e-88d3-4def-a78b-2090e952b8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:30.570593 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57267e-88d3-4def-a78b-2090e952b8ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:30.571413 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57267e-88d3-4def-a78b-2090e952b8ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:18:30.571793 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57267e-88d3-4def-a78b-2090e952b8ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:30.573673 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57267e-88d3-4def-a78b-2090e952b8ad admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:30.586110 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1067/2145] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:30.590008 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b0a20-406b-4a6a-ad43-0113d01b1fca admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:30.590706 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1079/2146] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:18:30.594836 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57267e-88d3-4def-a78b-2090e952b8ad admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:30.610958 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57267e-88d3-4def-a78b-2090e952b8ad admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:30.612049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b0a20-406b-4a6a-ad43-0113d01b1fca admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:30.618064 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa188e-d8e7-4b84-a359-26dc8ce1bd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:30.619067 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf9964-bfbe-4f10-943b-c2eec0f7a403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:30.619768 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa188e-d8e7-4b84-a359-26dc8ce1bd9e admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:30.619897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf9964-bfbe-4f10-943b-c2eec0f7a403 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:30.620120 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf9964-bfbe-4f10-943b-c2eec0f7a403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:30.620396 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf9964-bfbe-4f10-943b-c2eec0f7a403 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:30.620604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa188e-d8e7-4b84-a359-26dc8ce1bd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:30.621099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf9964-bfbe-4f10-943b-c2eec0f7a403 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:30.621282 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa188e-d8e7-4b84-a359-26dc8ce1bd9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:30.622409 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa188e-d8e7-4b84-a359-26dc8ce1bd9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:30.623812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf9964-bfbe-4f10-943b-c2eec0f7a403 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:30.624526 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa188e-d8e7-4b84-a359-26dc8ce1bd9e admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:30.636767 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1068/2147] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:30.646669 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa188e-d8e7-4b84-a359-26dc8ce1bd9e admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:30.656861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa188e-d8e7-4b84-a359-26dc8ce1bd9e admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:30.661586 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd089bf7-985e-42cc-9732-1250e8dddfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:30.663013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd089bf7-985e-42cc-9732-1250e8dddfab admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:30.663436 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd089bf7-985e-42cc-9732-1250e8dddfab admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:30.663699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd089bf7-985e-42cc-9732-1250e8dddfab admin admin] PATH_INFO: `/v3/projects/985b8e5a1e1d479da384b6d8868a9e98/users/b9bbb983be4b45f6a4f70c5a92b3550e/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:30.685001 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd089bf7-985e-42cc-9732-1250e8dddfab admin admin] RBAC: Authorizing `identity:create_grant(project_id=985b8e5a1e1d479da384b6d8868a9e98, user_id=b9bbb983be4b45f6a4f70c5a92b3550e, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:30.688607 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd089bf7-985e-42cc-9732-1250e8dddfab admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:30.751795 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1080/2148] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:30 2022] POST /identity/v3/projects => generated 391 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:18:30.761310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf9964-bfbe-4f10-943b-c2eec0f7a403 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:30.785132 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1069/2149] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:30 2022] PUT /identity/v3/projects/985b8e5a1e1d479da384b6d8868a9e98/users/b9bbb983be4b45f6a4f70c5a92b3550e/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:30.799360 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb46353-79f0-44c2-a10d-f74d83fba33f None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:30.799634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb46353-79f0-44c2-a10d-f74d83fba33f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:30.799867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb46353-79f0-44c2-a10d-f74d83fba33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:30.842500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-cdb46353-79f0-44c2-a10d-f74d83fba33f None None] MFA Rules not processed for user `b9bbb983be4b45f6a4f70c5a92b3550e`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:30.844693 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-cdb46353-79f0-44c2-a10d-f74d83fba33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:30.988355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf9964-bfbe-4f10-943b-c2eec0f7a403 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:31.067300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1070/2150] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:31.074651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb46353-79f0-44c2-a10d-f74d83fba33f None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:31.215235 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58930b-8305-4710-a108-605f65ac9281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:31.219336 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58930b-8305-4710-a108-605f65ac9281 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:31.219859 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58930b-8305-4710-a108-605f65ac9281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:31.221249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58930b-8305-4710-a108-605f65ac9281 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:31.222624 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58930b-8305-4710-a108-605f65ac9281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:31.228246 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58930b-8305-4710-a108-605f65ac9281 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:31.258554 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb46353-79f0-44c2-a10d-f74d83fba33f None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:31.265656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bef08-f94f-4836-946e-4aa93b6212af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:31.266806 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bef08-f94f-4836-946e-4aa93b6212af admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:31.267152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bef08-f94f-4836-946e-4aa93b6212af admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:31.269433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bef08-f94f-4836-946e-4aa93b6212af admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:31.269433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bef08-f94f-4836-946e-4aa93b6212af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:31.271534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bef08-f94f-4836-946e-4aa93b6212af admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:31.360208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1081/2151] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:30 2022] POST /identity/v3/users => generated 296 bytes in 599 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:31.364368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58930b-8305-4710-a108-605f65ac9281 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:31.377816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58930b-8305-4710-a108-605f65ac9281 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:31.386912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c61f2-4096-4ec3-ade9-af3531ac92ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:31.388068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c61f2-4096-4ec3-ade9-af3531ac92ea admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:31.388765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c61f2-4096-4ec3-ade9-af3531ac92ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:31.388765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c61f2-4096-4ec3-ade9-af3531ac92ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:31.389794 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c61f2-4096-4ec3-ade9-af3531ac92ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:31.391994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c61f2-4096-4ec3-ade9-af3531ac92ea admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:31.408329 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1082/2152] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:18:31.420292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c61f2-4096-4ec3-ade9-af3531ac92ea admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:31.441830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c61f2-4096-4ec3-ade9-af3531ac92ea admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:31.449516 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee78fb-80a4-4190-8b9e-7b312456296f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:31.450493 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee78fb-80a4-4190-8b9e-7b312456296f admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:31.450786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee78fb-80a4-4190-8b9e-7b312456296f admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:31.451067 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee78fb-80a4-4190-8b9e-7b312456296f admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:31.451874 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee78fb-80a4-4190-8b9e-7b312456296f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:31.453497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee78fb-80a4-4190-8b9e-7b312456296f admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:31.466736 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1071/2153] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:31 2022] POST /identity/v3/projects => generated 383 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:18:31.470588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bef08-f94f-4836-946e-4aa93b6212af admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:31.477833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1083/2154] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:31.483153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee78fb-80a4-4190-8b9e-7b312456296f admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:31.505491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee78fb-80a4-4190-8b9e-7b312456296f admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:31.651720 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bef08-f94f-4836-946e-4aa93b6212af admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:31.666169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602d49a-e0f5-4f39-b6f9-e0e1a2df3d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:31.666962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602d49a-e0f5-4f39-b6f9-e0e1a2df3d14 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:31.667247 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602d49a-e0f5-4f39-b6f9-e0e1a2df3d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:31.667598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602d49a-e0f5-4f39-b6f9-e0e1a2df3d14 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:31.668316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602d49a-e0f5-4f39-b6f9-e0e1a2df3d14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:31.672650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602d49a-e0f5-4f39-b6f9-e0e1a2df3d14 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:31.695744 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e481c7-63c7-46c1-b78d-cb34dbd262d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:31.696982 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e481c7-63c7-46c1-b78d-cb34dbd262d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:31.704174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e481c7-63c7-46c1-b78d-cb34dbd262d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:31.704966 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e481c7-63c7-46c1-b78d-cb34dbd262d7 admin admin] PATH_INFO: `/v3/projects/b2d2cad74b8442038d729add08410df8/users/79c6bd7b86cd41ae8a46e8abe86a3a1d/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:31.754442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e481c7-63c7-46c1-b78d-cb34dbd262d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2d2cad74b8442038d729add08410df8, user_id=79c6bd7b86cd41ae8a46e8abe86a3a1d, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:31.758262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e481c7-63c7-46c1-b78d-cb34dbd262d7 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:31.791295 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1072/2155] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:31 2022] POST /identity/v3/users => generated 298 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:18:31.792984 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602d49a-e0f5-4f39-b6f9-e0e1a2df3d14 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:31.827431 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602d49a-e0f5-4f39-b6f9-e0e1a2df3d14 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:31.870849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953484f-60de-484f-90ee-013f096fc771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:31.871664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953484f-60de-484f-90ee-013f096fc771 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:31.872036 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953484f-60de-484f-90ee-013f096fc771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:31.872416 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953484f-60de-484f-90ee-013f096fc771 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:31.873385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953484f-60de-484f-90ee-013f096fc771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:31.875326 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953484f-60de-484f-90ee-013f096fc771 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:31.889693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1084/2156] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:31 2022] PUT /identity/v3/projects/b2d2cad74b8442038d729add08410df8/users/79c6bd7b86cd41ae8a46e8abe86a3a1d/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:31.901723 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36e902-84c9-4872-99fd-a97d3787ea13 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:31.901723 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36e902-84c9-4872-99fd-a97d3787ea13 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:31.901920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36e902-84c9-4872-99fd-a97d3787ea13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:31.909962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1073/2157] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:31.911409 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953484f-60de-484f-90ee-013f096fc771 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:31.930781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953484f-60de-484f-90ee-013f096fc771 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:31.966217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-2a36e902-84c9-4872-99fd-a97d3787ea13 None None] MFA Rules not processed for user `79c6bd7b86cd41ae8a46e8abe86a3a1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:31.968728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-2a36e902-84c9-4872-99fd-a97d3787ea13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:32.083945 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2f8f1-da24-4e41-aa4f-c109127eacc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:32.085368 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2f8f1-da24-4e41-aa4f-c109127eacc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:32.085670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2f8f1-da24-4e41-aa4f-c109127eacc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:32.085928 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2f8f1-da24-4e41-aa4f-c109127eacc4 admin admin] PATH_INFO: `/v3/projects/0e6633ba8ffa4ccd94c17013cc056214/users/2be3ba54cd0847678cd7a2df6dd0e9d4/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:32.123772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2f8f1-da24-4e41-aa4f-c109127eacc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e6633ba8ffa4ccd94c17013cc056214, user_id=2be3ba54cd0847678cd7a2df6dd0e9d4, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:32.126832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2f8f1-da24-4e41-aa4f-c109127eacc4 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:32.147130 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1085/2158] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:18:32.153313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36e902-84c9-4872-99fd-a97d3787ea13 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:32.171593 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36e902-84c9-4872-99fd-a97d3787ea13 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:32.191594 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385de940-d56a-4571-941f-2afca304a707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:32.192378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385de940-d56a-4571-941f-2afca304a707 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:32.192673 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385de940-d56a-4571-941f-2afca304a707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:32.192952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385de940-d56a-4571-941f-2afca304a707 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:32.193760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385de940-d56a-4571-941f-2afca304a707 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:32.195722 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385de940-d56a-4571-941f-2afca304a707 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:32.216639 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1074/2159] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:31 2022] PUT /identity/v3/projects/0e6633ba8ffa4ccd94c17013cc056214/users/2be3ba54cd0847678cd7a2df6dd0e9d4/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:32.220628 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2f8f1-da24-4e41-aa4f-c109127eacc4 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:32.305764 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1086/2160] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:32 2022] POST /identity/v3/projects => generated 393 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:32.305764 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385de940-d56a-4571-941f-2afca304a707 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:32.480395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2f8f1-da24-4e41-aa4f-c109127eacc4 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:32.521827 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385de940-d56a-4571-941f-2afca304a707 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:32.531520 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34f371-eb9b-4b1f-a873-f50af0bc96c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:32.532202 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34f371-eb9b-4b1f-a873-f50af0bc96c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:32.532482 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34f371-eb9b-4b1f-a873-f50af0bc96c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:32.532758 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34f371-eb9b-4b1f-a873-f50af0bc96c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:32.533463 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34f371-eb9b-4b1f-a873-f50af0bc96c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:32.551114 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34f371-eb9b-4b1f-a873-f50af0bc96c1 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:32.608902 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66600d7e-9636-4a0a-a36a-02acf756f244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:32.614726 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66600d7e-9636-4a0a-a36a-02acf756f244 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:32.626848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66600d7e-9636-4a0a-a36a-02acf756f244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:32.626945 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66600d7e-9636-4a0a-a36a-02acf756f244 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:32.628951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66600d7e-9636-4a0a-a36a-02acf756f244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:32.633160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66600d7e-9636-4a0a-a36a-02acf756f244 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:32.658076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1075/2161] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:32.659183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66600d7e-9636-4a0a-a36a-02acf756f244 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:32.676804 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66600d7e-9636-4a0a-a36a-02acf756f244 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:32.709894 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1087/2162] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:32 2022] POST /identity/v3/users => generated 303 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:18:32.710913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34f371-eb9b-4b1f-a873-f50af0bc96c1 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:32.723683 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dab2df-93c2-4e1e-bb40-697ba88d4b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:32.724584 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dab2df-93c2-4e1e-bb40-697ba88d4b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:32.725873 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dab2df-93c2-4e1e-bb40-697ba88d4b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:32.726370 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dab2df-93c2-4e1e-bb40-697ba88d4b92 admin admin] PATH_INFO: `/v3/projects/0e6633ba8ffa4ccd94c17013cc056214/users/2be3ba54cd0847678cd7a2df6dd0e9d4/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:32.736714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dab2df-93c2-4e1e-bb40-697ba88d4b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e6633ba8ffa4ccd94c17013cc056214, user_id=2be3ba54cd0847678cd7a2df6dd0e9d4, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:32.744887 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34f371-eb9b-4b1f-a873-f50af0bc96c1 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:32.749421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dab2df-93c2-4e1e-bb40-697ba88d4b92 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:32.761218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4b126-66ef-4aab-bd49-955fa7f7318d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:32.761895 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4b126-66ef-4aab-bd49-955fa7f7318d admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:32.762208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4b126-66ef-4aab-bd49-955fa7f7318d admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:32.766982 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4b126-66ef-4aab-bd49-955fa7f7318d admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:32.769412 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4b126-66ef-4aab-bd49-955fa7f7318d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:32.770311 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4b126-66ef-4aab-bd49-955fa7f7318d admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:32.798130 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1076/2163] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:32 2022] PUT /identity/v3/projects/0e6633ba8ffa4ccd94c17013cc056214/users/2be3ba54cd0847678cd7a2df6dd0e9d4/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:32.804148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a031f7-9858-4005-8255-c9a95ec3cff8 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:32.819399 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a031f7-9858-4005-8255-c9a95ec3cff8 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:32.819399 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a031f7-9858-4005-8255-c9a95ec3cff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:32.819677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1088/2164] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:32.825074 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4b126-66ef-4aab-bd49-955fa7f7318d admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:32.846859 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4b126-66ef-4aab-bd49-955fa7f7318d admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:32.865437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-b7a031f7-9858-4005-8255-c9a95ec3cff8 None None] MFA Rules not processed for user `2be3ba54cd0847678cd7a2df6dd0e9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:32.867337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b7a031f7-9858-4005-8255-c9a95ec3cff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:32.956716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1077/2165] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:32.973381 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a031f7-9858-4005-8255-c9a95ec3cff8 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:32.976339 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b7a031f7-9858-4005-8255-c9a95ec3cff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:33.026557 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18632a8-f73c-472a-bea5-c31d6ecb640e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:33.027444 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18632a8-f73c-472a-bea5-c31d6ecb640e admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:33.027727 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18632a8-f73c-472a-bea5-c31d6ecb640e admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:33.027999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18632a8-f73c-472a-bea5-c31d6ecb640e admin admin] PATH_INFO: `/v3/projects/fabb8f902d1d4fe59fac437216e9b61e/users/7cc2530d7b774a88aad0f82faf23e2ad/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:33.046461 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18632a8-f73c-472a-bea5-c31d6ecb640e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fabb8f902d1d4fe59fac437216e9b61e, user_id=7cc2530d7b774a88aad0f82faf23e2ad, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:33.049119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18632a8-f73c-472a-bea5-c31d6ecb640e admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:33.132861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a031f7-9858-4005-8255-c9a95ec3cff8 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:33.140696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748a9f8-c275-4c0b-b558-a5f282b1ae86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:33.150289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748a9f8-c275-4c0b-b558-a5f282b1ae86 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:33.150289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748a9f8-c275-4c0b-b558-a5f282b1ae86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:33.150289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748a9f8-c275-4c0b-b558-a5f282b1ae86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:33.150829 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1089/2166] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:32 2022] PUT /identity/v3/projects/fabb8f902d1d4fe59fac437216e9b61e/users/7cc2530d7b774a88aad0f82faf23e2ad/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:33.153989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18632a8-f73c-472a-bea5-c31d6ecb640e admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:33.166259 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748a9f8-c275-4c0b-b558-a5f282b1ae86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:33.168642 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748a9f8-c275-4c0b-b558-a5f282b1ae86 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:33.185322 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1078/2167] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:18:33.284454 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18632a8-f73c-472a-bea5-c31d6ecb640e admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:33.289476 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39016196-614c-4c77-8fbb-85acab28695e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:33.290483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39016196-614c-4c77-8fbb-85acab28695e admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:33.290483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39016196-614c-4c77-8fbb-85acab28695e admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:33.291051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39016196-614c-4c77-8fbb-85acab28695e admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:33.291825 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39016196-614c-4c77-8fbb-85acab28695e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:33.293488 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39016196-614c-4c77-8fbb-85acab28695e admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:33.309312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1090/2168] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:18:33.310709 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748a9f8-c275-4c0b-b558-a5f282b1ae86 service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:33.323400 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748a9f8-c275-4c0b-b558-a5f282b1ae86 service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:33.328539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381b17e-c7cd-4c82-a77b-529b4a9d516d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:33.329496 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381b17e-c7cd-4c82-a77b-529b4a9d516d admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:33.329775 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381b17e-c7cd-4c82-a77b-529b4a9d516d admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:33.330044 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381b17e-c7cd-4c82-a77b-529b4a9d516d admin admin] PATH_INFO: `/v3/projects/fabb8f902d1d4fe59fac437216e9b61e/users/7cc2530d7b774a88aad0f82faf23e2ad/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:33.332356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381b17e-c7cd-4c82-a77b-529b4a9d516d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fabb8f902d1d4fe59fac437216e9b61e, user_id=7cc2530d7b774a88aad0f82faf23e2ad, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:33.335236 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381b17e-c7cd-4c82-a77b-529b4a9d516d admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:33.363248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1079/2169] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:33 2022] PUT /identity/v3/projects/fabb8f902d1d4fe59fac437216e9b61e/users/7cc2530d7b774a88aad0f82faf23e2ad/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:33.377206 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74f210-6e4d-4b2a-a785-5341dbda461c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:33.377558 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74f210-6e4d-4b2a-a785-5341dbda461c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:33.377802 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74f210-6e4d-4b2a-a785-5341dbda461c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:33.413488 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381b17e-c7cd-4c82-a77b-529b4a9d516d admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:33.468551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-5d74f210-6e4d-4b2a-a785-5341dbda461c None None] MFA Rules not processed for user `7cc2530d7b774a88aad0f82faf23e2ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:33.468551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-5d74f210-6e4d-4b2a-a785-5341dbda461c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:33.554573 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381b17e-c7cd-4c82-a77b-529b4a9d516d admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:33.563483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ca428-534c-49c2-8a2a-605a3097afdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:33.564614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ca428-534c-49c2-8a2a-605a3097afdd service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:33.565157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ca428-534c-49c2-8a2a-605a3097afdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:33.565621 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ca428-534c-49c2-8a2a-605a3097afdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:33.652297 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1091/2170] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:33 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:33.666704 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ca428-534c-49c2-8a2a-605a3097afdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:33.668938 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ca428-534c-49c2-8a2a-605a3097afdd service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:33.693771 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1080/2171] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:33.724070 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74f210-6e4d-4b2a-a785-5341dbda461c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:33.736360 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74f210-6e4d-4b2a-a785-5341dbda461c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:33.760112 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e14de-794e-408d-bb19-a1d22490a48b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:33.761168 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e14de-794e-408d-bb19-a1d22490a48b service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:33.761456 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e14de-794e-408d-bb19-a1d22490a48b service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:33.761764 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e14de-794e-408d-bb19-a1d22490a48b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:33.779134 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233e14de-794e-408d-bb19-a1d22490a48b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:33.781215 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233e14de-794e-408d-bb19-a1d22490a48b service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:33.795591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1092/2172] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:33.884189 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ca428-534c-49c2-8a2a-605a3097afdd service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:33.895617 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ca428-534c-49c2-8a2a-605a3097afdd service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:33.904599 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14879bd4-d3b1-4d77-81e6-79a7344c8e98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:33.906623 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14879bd4-d3b1-4d77-81e6-79a7344c8e98 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:33.906623 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14879bd4-d3b1-4d77-81e6-79a7344c8e98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:33.906623 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14879bd4-d3b1-4d77-81e6-79a7344c8e98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:34.004375 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14879bd4-d3b1-4d77-81e6-79a7344c8e98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:34.009192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14879bd4-d3b1-4d77-81e6-79a7344c8e98 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:34.026982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1081/2173] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:34.236567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb89b75-00c7-4615-aa38-487be5cc0e7c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:34.237065 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb89b75-00c7-4615-aa38-487be5cc0e7c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:34.237376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb89b75-00c7-4615-aa38-487be5cc0e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:34.264467 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-5bb89b75-00c7-4615-aa38-487be5cc0e7c None None] MFA Rules not processed for user `261d01f5b22340a09708c71be64edae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:34.265825 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-5bb89b75-00c7-4615-aa38-487be5cc0e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:34.334918 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1093/2174] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:34.571987 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14879bd4-d3b1-4d77-81e6-79a7344c8e98 service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:34.661839 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14879bd4-d3b1-4d77-81e6-79a7344c8e98 service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:34.674899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ea39f-1257-4dec-86ed-eebd4b5203d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:34.676118 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ea39f-1257-4dec-86ed-eebd4b5203d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:34.676426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ea39f-1257-4dec-86ed-eebd4b5203d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:34.677108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ea39f-1257-4dec-86ed-eebd4b5203d8 admin admin] PATH_INFO: `/v3/users/b9bbb983be4b45f6a4f70c5a92b3550e` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:34.679249 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ea39f-1257-4dec-86ed-eebd4b5203d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9bbb983be4b45f6a4f70c5a92b3550e)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:34.681339 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ea39f-1257-4dec-86ed-eebd4b5203d8 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:34.900965 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-7a5ea39f-1257-4dec-86ed-eebd4b5203d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9bbb983be4b45f6a4f70c5a92b3550e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:34.901735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-7a5ea39f-1257-4dec-86ed-eebd4b5203d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9bbb983be4b45f6a4f70c5a92b3550e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:34.954107 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-7a5ea39f-1257-4dec-86ed-eebd4b5203d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9bbb983be4b45f6a4f70c5a92b3550e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:34.954973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-7a5ea39f-1257-4dec-86ed-eebd4b5203d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9bbb983be4b45f6a4f70c5a92b3550e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:34.967541 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-7a5ea39f-1257-4dec-86ed-eebd4b5203d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9bbb983be4b45f6a4f70c5a92b3550e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:35.060014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1082/2175] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:34 2022] DELETE /identity/v3/users/b9bbb983be4b45f6a4f70c5a92b3550e => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:35.066172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db66d41-7316-4bbe-8ea6-02c35b850a1a None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:35.066172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db66d41-7316-4bbe-8ea6-02c35b850a1a None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:35.066172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db66d41-7316-4bbe-8ea6-02c35b850a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:35.123748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-6db66d41-7316-4bbe-8ea6-02c35b850a1a None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:35.125241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-6db66d41-7316-4bbe-8ea6-02c35b850a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:35.235166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1094/2176] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:35.251686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ea39f-1257-4dec-86ed-eebd4b5203d8 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:35.254867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-7a5ea39f-1257-4dec-86ed-eebd4b5203d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:35.356870 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ea39f-1257-4dec-86ed-eebd4b5203d8 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:35.361321 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df49c7b-f0f6-40fc-aa25-2aa868af549f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:35.362152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df49c7b-f0f6-40fc-aa25-2aa868af549f service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:35.362510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df49c7b-f0f6-40fc-aa25-2aa868af549f service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:35.363804 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df49c7b-f0f6-40fc-aa25-2aa868af549f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:35.401946 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df49c7b-f0f6-40fc-aa25-2aa868af549f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:35.403788 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df49c7b-f0f6-40fc-aa25-2aa868af549f service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:35.435085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1083/2177] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:35.918204 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db66d41-7316-4bbe-8ea6-02c35b850a1a None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:35.918204 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-6db66d41-7316-4bbe-8ea6-02c35b850a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:35.932884 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db66d41-7316-4bbe-8ea6-02c35b850a1a None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:35.938496 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0296cd7-5eb4-457d-985c-63a27ab69d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:35.939422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0296cd7-5eb4-457d-985c-63a27ab69d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:35.939697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0296cd7-5eb4-457d-985c-63a27ab69d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:35.940032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0296cd7-5eb4-457d-985c-63a27ab69d94 admin admin] PATH_INFO: `/v3/projects/985b8e5a1e1d479da384b6d8868a9e98` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:35.941180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0296cd7-5eb4-457d-985c-63a27ab69d94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=985b8e5a1e1d479da384b6d8868a9e98)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:35.946920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0296cd7-5eb4-457d-985c-63a27ab69d94 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:36.059897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-d0296cd7-5eb4-457d-985c-63a27ab69d94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '985b8e5a1e1d479da384b6d8868a9e98', 'request_id': 'req-d0296cd7-5eb4-457d-985c-63a27ab69d94'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:36.151279 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-d0296cd7-5eb4-457d-985c-63a27ab69d94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '985b8e5a1e1d479da384b6d8868a9e98', 'request_id': 'req-d0296cd7-5eb4-457d-985c-63a27ab69d94'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:36.161562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-d0296cd7-5eb4-457d-985c-63a27ab69d94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '985b8e5a1e1d479da384b6d8868a9e98', 'request_id': 'req-d0296cd7-5eb4-457d-985c-63a27ab69d94'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:36.174881 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-d0296cd7-5eb4-457d-985c-63a27ab69d94 admin admin] The token cache is being invalidate because project 985b8e5a1e1d479da384b6d8868a9e98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:36.175263 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-d0296cd7-5eb4-457d-985c-63a27ab69d94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:36.175988 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-d0296cd7-5eb4-457d-985c-63a27ab69d94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:36.177722 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1095/2178] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:35 2022] DELETE /identity/v3/projects/985b8e5a1e1d479da384b6d8868a9e98 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:36.182651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df49c7b-f0f6-40fc-aa25-2aa868af549f service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:36.210248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df49c7b-f0f6-40fc-aa25-2aa868af549f service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:36.276214 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b4095-0522-4c9d-ac2c-bbc655c79ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:36.277140 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b4095-0522-4c9d-ac2c-bbc655c79ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:36.277439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b4095-0522-4c9d-ac2c-bbc655c79ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:36.277721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b4095-0522-4c9d-ac2c-bbc655c79ae2 admin admin] PATH_INFO: `/v3/users/2be3ba54cd0847678cd7a2df6dd0e9d4` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:36.278970 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b4095-0522-4c9d-ac2c-bbc655c79ae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2be3ba54cd0847678cd7a2df6dd0e9d4)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:36.280974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b4095-0522-4c9d-ac2c-bbc655c79ae2 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:36.533049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-e72b4095-0522-4c9d-ac2c-bbc655c79ae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2be3ba54cd0847678cd7a2df6dd0e9d4'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:36.533049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-e72b4095-0522-4c9d-ac2c-bbc655c79ae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2be3ba54cd0847678cd7a2df6dd0e9d4'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:36.575610 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-e72b4095-0522-4c9d-ac2c-bbc655c79ae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2be3ba54cd0847678cd7a2df6dd0e9d4'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:36.590672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-e72b4095-0522-4c9d-ac2c-bbc655c79ae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2be3ba54cd0847678cd7a2df6dd0e9d4'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:36.600125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-e72b4095-0522-4c9d-ac2c-bbc655c79ae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2be3ba54cd0847678cd7a2df6dd0e9d4'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:36.713300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1084/2179] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:36 2022] DELETE /identity/v3/users/2be3ba54cd0847678cd7a2df6dd0e9d4 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:37.202265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0296cd7-5eb4-457d-985c-63a27ab69d94 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:37.202823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-d0296cd7-5eb4-457d-985c-63a27ab69d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:37.216135 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0296cd7-5eb4-457d-985c-63a27ab69d94 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:37.313227 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1842847-c7be-4411-bef8-d7e84495f2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:37.317237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1842847-c7be-4411-bef8-d7e84495f2c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:37.318203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1842847-c7be-4411-bef8-d7e84495f2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:37.318496 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1842847-c7be-4411-bef8-d7e84495f2c0 admin admin] PATH_INFO: `/v3/projects/0e6633ba8ffa4ccd94c17013cc056214` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:37.321400 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1842847-c7be-4411-bef8-d7e84495f2c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e6633ba8ffa4ccd94c17013cc056214)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:37.326419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1842847-c7be-4411-bef8-d7e84495f2c0 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:37.528351 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c1842847-c7be-4411-bef8-d7e84495f2c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e6633ba8ffa4ccd94c17013cc056214', 'request_id': 'req-c1842847-c7be-4411-bef8-d7e84495f2c0'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:37.619589 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c1842847-c7be-4411-bef8-d7e84495f2c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e6633ba8ffa4ccd94c17013cc056214', 'request_id': 'req-c1842847-c7be-4411-bef8-d7e84495f2c0'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:37.627727 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c1842847-c7be-4411-bef8-d7e84495f2c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e6633ba8ffa4ccd94c17013cc056214', 'request_id': 'req-c1842847-c7be-4411-bef8-d7e84495f2c0'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:37.658447 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c1842847-c7be-4411-bef8-d7e84495f2c0 admin admin] The token cache is being invalidate because project 0e6633ba8ffa4ccd94c17013cc056214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:37.659061 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c1842847-c7be-4411-bef8-d7e84495f2c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:37.659893 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c1842847-c7be-4411-bef8-d7e84495f2c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:37.661935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1096/2180] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:37 2022] DELETE /identity/v3/projects/0e6633ba8ffa4ccd94c17013cc056214 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:37.667151 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e29190-38d4-455e-bfae-8b6b3cda6e67 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:37.667680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e29190-38d4-455e-bfae-8b6b3cda6e67 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:37.667680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e29190-38d4-455e-bfae-8b6b3cda6e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:37.692687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-11e29190-38d4-455e-bfae-8b6b3cda6e67 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:37.695141 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-11e29190-38d4-455e-bfae-8b6b3cda6e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:37.798317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1085/2181] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:37.807512 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932926ff-8221-468a-a846-a444c950d919 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:37.808045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932926ff-8221-468a-a846-a444c950d919 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:37.808518 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932926ff-8221-468a-a846-a444c950d919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:37.845133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-932926ff-8221-468a-a846-a444c950d919 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:37.846527 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-932926ff-8221-468a-a846-a444c950d919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:37.856192 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1097/2182] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:37.860508 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e29190-38d4-455e-bfae-8b6b3cda6e67 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:37.871425 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e29190-38d4-455e-bfae-8b6b3cda6e67 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:37.877047 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b56ce-04b7-442c-8629-94be41667c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:37.877829 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b56ce-04b7-442c-8629-94be41667c36 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:37.878460 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b56ce-04b7-442c-8629-94be41667c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:37.878913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b56ce-04b7-442c-8629-94be41667c36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:37.879873 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b56ce-04b7-442c-8629-94be41667c36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:18:37.880581 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b56ce-04b7-442c-8629-94be41667c36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:37.883468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b56ce-04b7-442c-8629-94be41667c36 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:37.907861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1086/2183] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:18:37.908869 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932926ff-8221-468a-a846-a444c950d919 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:37.922407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932926ff-8221-468a-a846-a444c950d919 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:37.939466 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe497e8d-ae9d-46d9-9383-a50b359bd733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:37.940626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe497e8d-ae9d-46d9-9383-a50b359bd733 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:37.941112 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe497e8d-ae9d-46d9-9383-a50b359bd733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:37.941722 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe497e8d-ae9d-46d9-9383-a50b359bd733 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:37.943202 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe497e8d-ae9d-46d9-9383-a50b359bd733 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:37.946223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe497e8d-ae9d-46d9-9383-a50b359bd733 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:38.055060 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b56ce-04b7-442c-8629-94be41667c36 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:38.055460 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1098/2184] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:37 2022] POST /identity/v3/projects => generated 361 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:18:38.065934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b56ce-04b7-442c-8629-94be41667c36 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:38.132290 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdfeaad-6892-488e-ac83-c06c7dcc2c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:38.132968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdfeaad-6892-488e-ac83-c06c7dcc2c27 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:38.133253 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdfeaad-6892-488e-ac83-c06c7dcc2c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:38.133529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdfeaad-6892-488e-ac83-c06c7dcc2c27 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:38.134264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdfeaad-6892-488e-ac83-c06c7dcc2c27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:38.136181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdfeaad-6892-488e-ac83-c06c7dcc2c27 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:38.248031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1087/2185] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:38 2022] POST /identity/v3/users => generated 281 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:38.248229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe497e8d-ae9d-46d9-9383-a50b359bd733 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:38.255016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe497e8d-ae9d-46d9-9383-a50b359bd733 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:38.260576 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8163f6-cdd9-4388-bf22-e9aa6124b267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:38.261362 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8163f6-cdd9-4388-bf22-e9aa6124b267 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:38.261861 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8163f6-cdd9-4388-bf22-e9aa6124b267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:38.262263 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8163f6-cdd9-4388-bf22-e9aa6124b267 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:38.263100 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8163f6-cdd9-4388-bf22-e9aa6124b267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:38.264786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8163f6-cdd9-4388-bf22-e9aa6124b267 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:38.278652 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1099/2186] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:38.282823 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdfeaad-6892-488e-ac83-c06c7dcc2c27 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:38.293718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdfeaad-6892-488e-ac83-c06c7dcc2c27 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:38.298473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149a8e6-ee1d-4629-8e36-1291ab5618c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:38.299125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149a8e6-ee1d-4629-8e36-1291ab5618c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:38.299407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149a8e6-ee1d-4629-8e36-1291ab5618c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:38.299677 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149a8e6-ee1d-4629-8e36-1291ab5618c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:38.300395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8149a8e6-ee1d-4629-8e36-1291ab5618c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:38.301995 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8149a8e6-ee1d-4629-8e36-1291ab5618c7 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:38.312747 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1088/2187] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:38.316963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8163f6-cdd9-4388-bf22-e9aa6124b267 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:38.328732 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8163f6-cdd9-4388-bf22-e9aa6124b267 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:38.333731 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66071342-8c79-49e5-9077-88354b7c57de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:38.334769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66071342-8c79-49e5-9077-88354b7c57de admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:38.335046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66071342-8c79-49e5-9077-88354b7c57de admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:38.336036 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66071342-8c79-49e5-9077-88354b7c57de admin admin] PATH_INFO: `/v3/projects/4acfedec8b1149c8bb3ce7fb7629dbd8/users/13bf587bcbe24180b7d055300a53ae8d/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:38.352447 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66071342-8c79-49e5-9077-88354b7c57de admin admin] RBAC: Authorizing `identity:create_grant(project_id=4acfedec8b1149c8bb3ce7fb7629dbd8, user_id=13bf587bcbe24180b7d055300a53ae8d, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:38.355587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66071342-8c79-49e5-9077-88354b7c57de admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:38.389444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149a8e6-ee1d-4629-8e36-1291ab5618c7 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:38.391571 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8149a8e6-ee1d-4629-8e36-1291ab5618c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:38.407213 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149a8e6-ee1d-4629-8e36-1291ab5618c7 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:38.412352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8008af4-c9e9-489a-a9d7-763942233ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:38.414069 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8008af4-c9e9-489a-a9d7-763942233ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:38.414518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8008af4-c9e9-489a-a9d7-763942233ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:38.414875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8008af4-c9e9-489a-a9d7-763942233ad6 admin admin] PATH_INFO: `/v3/users/4c3123bcc38749058b7e2e697d827c8a` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:38.416242 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8008af4-c9e9-489a-a9d7-763942233ad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c3123bcc38749058b7e2e697d827c8a)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:38.418321 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8008af4-c9e9-489a-a9d7-763942233ad6 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:38.458992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1100/2188] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:38 2022] PUT /identity/v3/projects/4acfedec8b1149c8bb3ce7fb7629dbd8/users/13bf587bcbe24180b7d055300a53ae8d/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:38.468000 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867357d-eddf-4432-9508-f79b14ff53c5 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:38.468513 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867357d-eddf-4432-9508-f79b14ff53c5 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:38.468948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867357d-eddf-4432-9508-f79b14ff53c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:38.497817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-3867357d-eddf-4432-9508-f79b14ff53c5 None None] MFA Rules not processed for user `13bf587bcbe24180b7d055300a53ae8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:38.499263 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3867357d-eddf-4432-9508-f79b14ff53c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:38.578653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1101/2189] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:18:38.612881 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867357d-eddf-4432-9508-f79b14ff53c5 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:38.614391 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3867357d-eddf-4432-9508-f79b14ff53c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:38.682925 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-e8008af4-c9e9-489a-a9d7-763942233ad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c3123bcc38749058b7e2e697d827c8a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:38.684136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-e8008af4-c9e9-489a-a9d7-763942233ad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c3123bcc38749058b7e2e697d827c8a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:38.698152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-e8008af4-c9e9-489a-a9d7-763942233ad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c3123bcc38749058b7e2e697d827c8a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:38.698487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867357d-eddf-4432-9508-f79b14ff53c5 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:38.699037 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-e8008af4-c9e9-489a-a9d7-763942233ad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c3123bcc38749058b7e2e697d827c8a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:38.708103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-e8008af4-c9e9-489a-a9d7-763942233ad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c3123bcc38749058b7e2e697d827c8a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:38.711671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844ce81-5b36-4cd1-95cf-f29f22c4107b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:38.713775 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844ce81-5b36-4cd1-95cf-f29f22c4107b service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:38.715605 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844ce81-5b36-4cd1-95cf-f29f22c4107b service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:38.715986 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844ce81-5b36-4cd1-95cf-f29f22c4107b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:38.720484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b844ce81-5b36-4cd1-95cf-f29f22c4107b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:38.749151 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844ce81-5b36-4cd1-95cf-f29f22c4107b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:38.752829 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844ce81-5b36-4cd1-95cf-f29f22c4107b service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:38.777288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1102/2190] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:38.795875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844ce81-5b36-4cd1-95cf-f29f22c4107b service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:38.804122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b844ce81-5b36-4cd1-95cf-f29f22c4107b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:38.816042 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1089/2191] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:38 2022] DELETE /identity/v3/users/4c3123bcc38749058b7e2e697d827c8a => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:38.821594 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10633c8d-a357-4698-b90b-9b8726345ea6 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:38.822216 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10633c8d-a357-4698-b90b-9b8726345ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:38.822734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10633c8d-a357-4698-b90b-9b8726345ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:38.869479 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-10633c8d-a357-4698-b90b-9b8726345ea6 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:38.871374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-10633c8d-a357-4698-b90b-9b8726345ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:38.970259 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1090/2192] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:18:38.976268 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10633c8d-a357-4698-b90b-9b8726345ea6 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:38.981881 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-10633c8d-a357-4698-b90b-9b8726345ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:38.985927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844ce81-5b36-4cd1-95cf-f29f22c4107b service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:39.000195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ba53b-7435-4843-9621-d7160280a266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:39.002289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ba53b-7435-4843-9621-d7160280a266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:39.002589 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ba53b-7435-4843-9621-d7160280a266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:39.007779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ba53b-7435-4843-9621-d7160280a266 admin admin] PATH_INFO: `/v3/users/13bf587bcbe24180b7d055300a53ae8d` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:39.007779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ba53b-7435-4843-9621-d7160280a266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13bf587bcbe24180b7d055300a53ae8d)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:39.014353 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ba53b-7435-4843-9621-d7160280a266 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:39.097361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10633c8d-a357-4698-b90b-9b8726345ea6 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:39.101919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29934799-da9e-4b40-9bed-cd1d3bcbbd80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:39.102819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29934799-da9e-4b40-9bed-cd1d3bcbbd80 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:39.103108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29934799-da9e-4b40-9bed-cd1d3bcbbd80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:39.103412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29934799-da9e-4b40-9bed-cd1d3bcbbd80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:39.113083 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29934799-da9e-4b40-9bed-cd1d3bcbbd80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:39.114603 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29934799-da9e-4b40-9bed-cd1d3bcbbd80 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:39.124352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1091/2193] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:18:39.296772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-9e5ba53b-7435-4843-9621-d7160280a266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13bf587bcbe24180b7d055300a53ae8d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:39.297438 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-9e5ba53b-7435-4843-9621-d7160280a266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13bf587bcbe24180b7d055300a53ae8d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:39.298052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-9e5ba53b-7435-4843-9621-d7160280a266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13bf587bcbe24180b7d055300a53ae8d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:39.311752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-9e5ba53b-7435-4843-9621-d7160280a266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13bf587bcbe24180b7d055300a53ae8d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:39.321108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-9e5ba53b-7435-4843-9621-d7160280a266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13bf587bcbe24180b7d055300a53ae8d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:39.424821 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1103/2194] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:38 2022] DELETE /identity/v3/users/13bf587bcbe24180b7d055300a53ae8d => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:39.427543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909f91b-4fc9-46cf-99fd-1827986141fd None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:39.428014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909f91b-4fc9-46cf-99fd-1827986141fd None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:39.428695 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909f91b-4fc9-46cf-99fd-1827986141fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:39.475707 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-8909f91b-4fc9-46cf-99fd-1827986141fd None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:39.477713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8909f91b-4fc9-46cf-99fd-1827986141fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:39.554871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1092/2195] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:18:39.565338 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ba53b-7435-4843-9621-d7160280a266 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:39.567265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-9e5ba53b-7435-4843-9621-d7160280a266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:39.629731 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ba53b-7435-4843-9621-d7160280a266 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:39.635051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444aa0c-74e9-4637-9df4-82007c5e1649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:39.635763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444aa0c-74e9-4637-9df4-82007c5e1649 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:39.636065 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444aa0c-74e9-4637-9df4-82007c5e1649 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:39.636349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444aa0c-74e9-4637-9df4-82007c5e1649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:39.647156 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444aa0c-74e9-4637-9df4-82007c5e1649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:39.648854 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444aa0c-74e9-4637-9df4-82007c5e1649 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:39.655577 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909f91b-4fc9-46cf-99fd-1827986141fd None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:39.657090 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8909f91b-4fc9-46cf-99fd-1827986141fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:39.662137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1104/2196] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:39.685644 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909f91b-4fc9-46cf-99fd-1827986141fd None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:39.693380 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633cc6c7-71e1-4d1a-bd24-1509f989db96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:39.714375 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cc6c7-71e1-4d1a-bd24-1509f989db96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:39.714375 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cc6c7-71e1-4d1a-bd24-1509f989db96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:39.714375 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cc6c7-71e1-4d1a-bd24-1509f989db96 admin admin] PATH_INFO: `/v3/projects/332e2b023d4d42508a455845d0f6e52f` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:39.714375 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633cc6c7-71e1-4d1a-bd24-1509f989db96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=332e2b023d4d42508a455845d0f6e52f)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:39.715140 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633cc6c7-71e1-4d1a-bd24-1509f989db96 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:39.909051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-633cc6c7-71e1-4d1a-bd24-1509f989db96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '332e2b023d4d42508a455845d0f6e52f', 'request_id': 'req-633cc6c7-71e1-4d1a-bd24-1509f989db96'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:39.949140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444aa0c-74e9-4637-9df4-82007c5e1649 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:39.951029 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4444aa0c-74e9-4637-9df4-82007c5e1649 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:39.968098 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444aa0c-74e9-4637-9df4-82007c5e1649 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:40.046690 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882a348-1e79-471a-a961-98690514f9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:40.047671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882a348-1e79-471a-a961-98690514f9c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:40.047961 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882a348-1e79-471a-a961-98690514f9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:40.048243 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882a348-1e79-471a-a961-98690514f9c6 admin admin] PATH_INFO: `/v3/users/79c6bd7b86cd41ae8a46e8abe86a3a1d` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:40.049486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882a348-1e79-471a-a961-98690514f9c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79c6bd7b86cd41ae8a46e8abe86a3a1d)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:40.051855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882a348-1e79-471a-a961-98690514f9c6 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:40.072064 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-633cc6c7-71e1-4d1a-bd24-1509f989db96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '332e2b023d4d42508a455845d0f6e52f', 'request_id': 'req-633cc6c7-71e1-4d1a-bd24-1509f989db96'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:40.079502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-633cc6c7-71e1-4d1a-bd24-1509f989db96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '332e2b023d4d42508a455845d0f6e52f', 'request_id': 'req-633cc6c7-71e1-4d1a-bd24-1509f989db96'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:40.100111 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-633cc6c7-71e1-4d1a-bd24-1509f989db96 admin admin] The token cache is being invalidate because project 332e2b023d4d42508a455845d0f6e52f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:40.100389 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-633cc6c7-71e1-4d1a-bd24-1509f989db96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:40.101295 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-633cc6c7-71e1-4d1a-bd24-1509f989db96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:40.103766 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1093/2197] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:39 2022] DELETE /identity/v3/projects/332e2b023d4d42508a455845d0f6e52f => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:40.109431 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca95519-b701-4a0c-a884-edd2036ca846 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:40.112810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca95519-b701-4a0c-a884-edd2036ca846 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:40.112810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca95519-b701-4a0c-a884-edd2036ca846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:40.134716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-7ca95519-b701-4a0c-a884-edd2036ca846 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:40.136323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-7ca95519-b701-4a0c-a884-edd2036ca846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:40.147648 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1094/2198] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:40.158562 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7916b5-5245-4585-b377-102bd277810b None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:40.158998 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7916b5-5245-4585-b377-102bd277810b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:40.159436 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7916b5-5245-4585-b377-102bd277810b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:40.194636 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-0b7916b5-5245-4585-b377-102bd277810b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:40.196498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0b7916b5-5245-4585-b377-102bd277810b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:40.217165 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1095/2199] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:40.218365 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7916b5-5245-4585-b377-102bd277810b None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:40.230552 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7916b5-5245-4585-b377-102bd277810b None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:40.237566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0f869-c3c8-4e2b-89cb-925e4b33aced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:40.238290 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0f869-c3c8-4e2b-89cb-925e4b33aced admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:40.238675 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0f869-c3c8-4e2b-89cb-925e4b33aced admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:40.238991 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0f869-c3c8-4e2b-89cb-925e4b33aced admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:40.239720 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0f869-c3c8-4e2b-89cb-925e4b33aced admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:18:40.240035 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0f869-c3c8-4e2b-89cb-925e4b33aced admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:40.242582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0f869-c3c8-4e2b-89cb-925e4b33aced admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:40.258531 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1096/2200] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:18:40.266024 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0f869-c3c8-4e2b-89cb-925e4b33aced admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:40.277654 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0f869-c3c8-4e2b-89cb-925e4b33aced admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:40.284543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a32be-d69a-4900-b40d-3f76c87720bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:40.285937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a32be-d69a-4900-b40d-3f76c87720bd admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:40.285937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a32be-d69a-4900-b40d-3f76c87720bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:40.289839 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a32be-d69a-4900-b40d-3f76c87720bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:40.290746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a32be-d69a-4900-b40d-3f76c87720bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:40.292686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a32be-d69a-4900-b40d-3f76c87720bd admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:40.352413 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6882a348-1e79-471a-a961-98690514f9c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79c6bd7b86cd41ae8a46e8abe86a3a1d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:40.353353 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6882a348-1e79-471a-a961-98690514f9c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79c6bd7b86cd41ae8a46e8abe86a3a1d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:40.354034 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6882a348-1e79-471a-a961-98690514f9c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79c6bd7b86cd41ae8a46e8abe86a3a1d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:40.369341 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6882a348-1e79-471a-a961-98690514f9c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79c6bd7b86cd41ae8a46e8abe86a3a1d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:40.377266 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6882a348-1e79-471a-a961-98690514f9c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79c6bd7b86cd41ae8a46e8abe86a3a1d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:40.437387 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1097/2201] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:40 2022] POST /identity/v3/projects => generated 373 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:40.437387 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a32be-d69a-4900-b40d-3f76c87720bd admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:40.438864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-806a32be-d69a-4900-b40d-3f76c87720bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:40.487440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1105/2202] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:39 2022] DELETE /identity/v3/users/79c6bd7b86cd41ae8a46e8abe86a3a1d => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:40.488711 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882a348-1e79-471a-a961-98690514f9c6 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:40.490534 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-6882a348-1e79-471a-a961-98690514f9c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:40.591639 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a32be-d69a-4900-b40d-3f76c87720bd admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:40.619022 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882a348-1e79-471a-a961-98690514f9c6 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:40.623852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e46d6-510d-4306-83bd-bb2fa565e834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:40.624644 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e46d6-510d-4306-83bd-bb2fa565e834 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:40.625025 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e46d6-510d-4306-83bd-bb2fa565e834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:40.625308 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e46d6-510d-4306-83bd-bb2fa565e834 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:40.626163 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e46d6-510d-4306-83bd-bb2fa565e834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:40.628669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e46d6-510d-4306-83bd-bb2fa565e834 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:40.672889 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eef3a4-1a6d-41f5-94b9-b5c260cf9788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:40.673888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eef3a4-1a6d-41f5-94b9-b5c260cf9788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:40.674277 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eef3a4-1a6d-41f5-94b9-b5c260cf9788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:40.674628 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eef3a4-1a6d-41f5-94b9-b5c260cf9788 admin admin] PATH_INFO: `/v3/projects/4acfedec8b1149c8bb3ce7fb7629dbd8` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:40.676038 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eef3a4-1a6d-41f5-94b9-b5c260cf9788 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4acfedec8b1149c8bb3ce7fb7629dbd8)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:40.678052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eef3a4-1a6d-41f5-94b9-b5c260cf9788 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:40.712137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1106/2203] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:40 2022] POST /identity/v3/users => generated 287 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:40.713569 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f591e-9d51-455a-9090-77da8a3c6b70 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:40.714168 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f591e-9d51-455a-9090-77da8a3c6b70 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:40.718656 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f591e-9d51-455a-9090-77da8a3c6b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:40.755628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-d81f591e-9d51-455a-9090-77da8a3c6b70 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:40.756990 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-d81f591e-9d51-455a-9090-77da8a3c6b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:40.774357 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1107/2204] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:40.774357 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f591e-9d51-455a-9090-77da8a3c6b70 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:40.786476 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-c0eef3a4-1a6d-41f5-94b9-b5c260cf9788 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4acfedec8b1149c8bb3ce7fb7629dbd8', 'request_id': 'req-c0eef3a4-1a6d-41f5-94b9-b5c260cf9788'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:40.792678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f591e-9d51-455a-9090-77da8a3c6b70 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:40.847667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-c0eef3a4-1a6d-41f5-94b9-b5c260cf9788 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4acfedec8b1149c8bb3ce7fb7629dbd8', 'request_id': 'req-c0eef3a4-1a6d-41f5-94b9-b5c260cf9788'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:40.857328 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-c0eef3a4-1a6d-41f5-94b9-b5c260cf9788 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4acfedec8b1149c8bb3ce7fb7629dbd8', 'request_id': 'req-c0eef3a4-1a6d-41f5-94b9-b5c260cf9788'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:40.884283 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-c0eef3a4-1a6d-41f5-94b9-b5c260cf9788 admin admin] The token cache is being invalidate because project 4acfedec8b1149c8bb3ce7fb7629dbd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:40.884902 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-c0eef3a4-1a6d-41f5-94b9-b5c260cf9788 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:40.885828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-c0eef3a4-1a6d-41f5-94b9-b5c260cf9788 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:40.887993 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1098/2205] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:40 2022] DELETE /identity/v3/projects/4acfedec8b1149c8bb3ce7fb7629dbd8 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:40.889343 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eef3a4-1a6d-41f5-94b9-b5c260cf9788 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:40.891129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c0eef3a4-1a6d-41f5-94b9-b5c260cf9788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:40.959161 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6365fbb-fc1f-4d83-ad9e-067bdcc93fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:40.960397 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6365fbb-fc1f-4d83-ad9e-067bdcc93fda admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:40.960928 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6365fbb-fc1f-4d83-ad9e-067bdcc93fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:40.961704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6365fbb-fc1f-4d83-ad9e-067bdcc93fda admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:40.962750 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6365fbb-fc1f-4d83-ad9e-067bdcc93fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:40.964926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6365fbb-fc1f-4d83-ad9e-067bdcc93fda admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:40.983890 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1108/2206] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:40.984042 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eef3a4-1a6d-41f5-94b9-b5c260cf9788 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:40.984172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a658e-6339-4ba6-8ab3-200f5f34012c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:40.984172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a658e-6339-4ba6-8ab3-200f5f34012c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:40.984172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a658e-6339-4ba6-8ab3-200f5f34012c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:40.987288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b381826-cd80-4c3b-9843-65e28a3349c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:40.988132 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b381826-cd80-4c3b-9843-65e28a3349c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:40.988500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b381826-cd80-4c3b-9843-65e28a3349c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:40.988849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b381826-cd80-4c3b-9843-65e28a3349c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:40.992689 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0b381826-cd80-4c3b-9843-65e28a3349c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:41.004497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-cb2a658e-6339-4ba6-8ab3-200f5f34012c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:41.005988 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-cb2a658e-6339-4ba6-8ab3-200f5f34012c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:41.015375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1109/2207] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:41.016288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a658e-6339-4ba6-8ab3-200f5f34012c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:41.017542 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-cb2a658e-6339-4ba6-8ab3-200f5f34012c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:41.027849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b381826-cd80-4c3b-9843-65e28a3349c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:41.029808 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b381826-cd80-4c3b-9843-65e28a3349c1 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:41.030874 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a658e-6339-4ba6-8ab3-200f5f34012c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:41.036974 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebfbb6-dc12-4b3b-91f1-09adad469b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:41.037869 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebfbb6-dc12-4b3b-91f1-09adad469b2b admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:41.038260 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebfbb6-dc12-4b3b-91f1-09adad469b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:41.039283 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebfbb6-dc12-4b3b-91f1-09adad469b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:41.044017 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebfbb6-dc12-4b3b-91f1-09adad469b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:41.044017 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebfbb6-dc12-4b3b-91f1-09adad469b2b admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:41.053807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1110/2209] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:41.053807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10180f-a5f4-4d1c-89c2-cfdc112f7038 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:41.053807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10180f-a5f4-4d1c-89c2-cfdc112f7038 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:41.053807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10180f-a5f4-4d1c-89c2-cfdc112f7038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:41.066203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1099/2209] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:41.067507 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b381826-cd80-4c3b-9843-65e28a3349c1 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:41.079189 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b381826-cd80-4c3b-9843-65e28a3349c1 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:41.086164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc44820-4fef-4d84-be00-6531f0a01894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:41.087615 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc44820-4fef-4d84-be00-6531f0a01894 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:41.088214 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc44820-4fef-4d84-be00-6531f0a01894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:41.088790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc44820-4fef-4d84-be00-6531f0a01894 admin admin] PATH_INFO: `/v3/projects/b9724c9a575141d48bbfbde685774622/users/f213f79341ab4462b3d51352ccb9de2c/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:41.114200 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-df10180f-a5f4-4d1c-89c2-cfdc112f7038 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:41.116444 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-df10180f-a5f4-4d1c-89c2-cfdc112f7038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:41.124865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc44820-4fef-4d84-be00-6531f0a01894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9724c9a575141d48bbfbde685774622, user_id=f213f79341ab4462b3d51352ccb9de2c, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:41.132181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc44820-4fef-4d84-be00-6531f0a01894 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:41.141294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1111/2210] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:41.145998 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10180f-a5f4-4d1c-89c2-cfdc112f7038 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:41.158314 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10180f-a5f4-4d1c-89c2-cfdc112f7038 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:41.165103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655f443-8bb0-444e-8906-356d54a3160f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:41.165855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655f443-8bb0-444e-8906-356d54a3160f admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:41.166564 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655f443-8bb0-444e-8906-356d54a3160f admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:41.166564 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655f443-8bb0-444e-8906-356d54a3160f admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:41.167689 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655f443-8bb0-444e-8906-356d54a3160f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:18:41.168215 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655f443-8bb0-444e-8906-356d54a3160f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:41.170145 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655f443-8bb0-444e-8906-356d54a3160f admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:41.192439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1112/2211] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:18:41.197727 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655f443-8bb0-444e-8906-356d54a3160f admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:41.219581 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655f443-8bb0-444e-8906-356d54a3160f admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:41.228539 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792347d8-730e-422f-88d2-62cf9d047e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:41.230337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792347d8-730e-422f-88d2-62cf9d047e3e admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:41.232892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792347d8-730e-422f-88d2-62cf9d047e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:41.238222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1100/2212] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:41 2022] PUT /identity/v3/projects/b9724c9a575141d48bbfbde685774622/users/f213f79341ab4462b3d51352ccb9de2c/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:41.242533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792347d8-730e-422f-88d2-62cf9d047e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:41.242533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792347d8-730e-422f-88d2-62cf9d047e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:41.242533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792347d8-730e-422f-88d2-62cf9d047e3e admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:41.244844 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bd53e-c5c9-4ea9-b090-cca5e865521b None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:41.245310 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bd53e-c5c9-4ea9-b090-cca5e865521b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:41.245701 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bd53e-c5c9-4ea9-b090-cca5e865521b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:41.278281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-0b2bd53e-c5c9-4ea9-b090-cca5e865521b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:41.279563 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0b2bd53e-c5c9-4ea9-b090-cca5e865521b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:41.406133 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1101/2213] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:41.416019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bd53e-c5c9-4ea9-b090-cca5e865521b None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:41.417430 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0b2bd53e-c5c9-4ea9-b090-cca5e865521b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:41.498100 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bd53e-c5c9-4ea9-b090-cca5e865521b None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:41.502623 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e422c75-28ab-40c5-a6f8-fbdcf05a0b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:41.503285 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e422c75-28ab-40c5-a6f8-fbdcf05a0b4b service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:41.503567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e422c75-28ab-40c5-a6f8-fbdcf05a0b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:41.504207 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e422c75-28ab-40c5-a6f8-fbdcf05a0b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:41.515289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e422c75-28ab-40c5-a6f8-fbdcf05a0b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:41.516985 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e422c75-28ab-40c5-a6f8-fbdcf05a0b4b service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:41.528941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1102/2214] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:18:41.970661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1113/2215] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:41 2022] POST /identity/v3/projects => generated 371 bytes in 774 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:41.974228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e422c75-28ab-40c5-a6f8-fbdcf05a0b4b service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:41.985043 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e422c75-28ab-40c5-a6f8-fbdcf05a0b4b service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:42.076312 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512c16c-1ef6-4799-9c33-c7857d6106de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:42.077338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512c16c-1ef6-4799-9c33-c7857d6106de admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:42.077956 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512c16c-1ef6-4799-9c33-c7857d6106de admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:42.078419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512c16c-1ef6-4799-9c33-c7857d6106de admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:42.079323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512c16c-1ef6-4799-9c33-c7857d6106de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:42.081320 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512c16c-1ef6-4799-9c33-c7857d6106de admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:42.197066 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792347d8-730e-422f-88d2-62cf9d047e3e admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:42.199072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-792347d8-730e-422f-88d2-62cf9d047e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:42.224324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792347d8-730e-422f-88d2-62cf9d047e3e admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:42.225111 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1103/2216] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:41 2022] POST /identity/v3/users => generated 286 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:42.229309 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512c16c-1ef6-4799-9c33-c7857d6106de admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:42.241113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f6c03-e539-4042-a256-830df1776ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:42.242556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f6c03-e539-4042-a256-830df1776ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:42.242832 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f6c03-e539-4042-a256-830df1776ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:42.243166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f6c03-e539-4042-a256-830df1776ded admin admin] PATH_INFO: `/v3/projects/b2d2cad74b8442038d729add08410df8` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:42.245897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f6c03-e539-4042-a256-830df1776ded admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2d2cad74b8442038d729add08410df8)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:42.247847 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f6c03-e539-4042-a256-830df1776ded admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:42.270209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512c16c-1ef6-4799-9c33-c7857d6106de admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:42.276924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6817d4-a872-456d-b323-4a154be5f4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:42.281598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6817d4-a872-456d-b323-4a154be5f4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:42.281778 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6817d4-a872-456d-b323-4a154be5f4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:42.282031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6817d4-a872-456d-b323-4a154be5f4a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:42.282824 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6817d4-a872-456d-b323-4a154be5f4a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:42.284507 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6817d4-a872-456d-b323-4a154be5f4a4 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:42.306950 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1104/2217] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:42.311189 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6817d4-a872-456d-b323-4a154be5f4a4 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:42.321989 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6817d4-a872-456d-b323-4a154be5f4a4 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:42.326953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c5e52-c024-4324-b13a-af098c8b88a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:42.327609 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c5e52-c024-4324-b13a-af098c8b88a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:42.327915 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c5e52-c024-4324-b13a-af098c8b88a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:42.328193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c5e52-c024-4324-b13a-af098c8b88a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:42.329076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c5e52-c024-4324-b13a-af098c8b88a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:42.330739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c5e52-c024-4324-b13a-af098c8b88a2 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:42.341458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1105/2218] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:18:42.345652 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c5e52-c024-4324-b13a-af098c8b88a2 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:42.356145 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c5e52-c024-4324-b13a-af098c8b88a2 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:42.360915 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2742a2-ba89-4ee3-9827-4f4d2f7895c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:42.361788 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2742a2-ba89-4ee3-9827-4f4d2f7895c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:42.362073 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2742a2-ba89-4ee3-9827-4f4d2f7895c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:42.362380 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2742a2-ba89-4ee3-9827-4f4d2f7895c4 admin admin] PATH_INFO: `/v3/projects/752cb97b116941b7bc7ef28b2e796b1f/users/e98e849891bf45768936056e3206d880/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:42.378216 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2742a2-ba89-4ee3-9827-4f4d2f7895c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=752cb97b116941b7bc7ef28b2e796b1f, user_id=e98e849891bf45768936056e3206d880, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:42.382147 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2742a2-ba89-4ee3-9827-4f4d2f7895c4 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:42.471918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1106/2219] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:42 2022] PUT /identity/v3/projects/752cb97b116941b7bc7ef28b2e796b1f/users/e98e849891bf45768936056e3206d880/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:42.477943 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cf02c-c5a2-4b96-9277-67bf65393928 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:42.478300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cf02c-c5a2-4b96-9277-67bf65393928 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:42.478577 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cf02c-c5a2-4b96-9277-67bf65393928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:42.483841 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a75f6c03-e539-4042-a256-830df1776ded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2d2cad74b8442038d729add08410df8', 'request_id': 'req-a75f6c03-e539-4042-a256-830df1776ded'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:42.497669 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-5f5cf02c-c5a2-4b96-9277-67bf65393928 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:42.506698 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-5f5cf02c-c5a2-4b96-9277-67bf65393928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:42.581375 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1107/2220] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:42.583292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a75f6c03-e539-4042-a256-830df1776ded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2d2cad74b8442038d729add08410df8', 'request_id': 'req-a75f6c03-e539-4042-a256-830df1776ded'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:42.588286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cf02c-c5a2-4b96-9277-67bf65393928 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:42.593703 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a75f6c03-e539-4042-a256-830df1776ded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2d2cad74b8442038d729add08410df8', 'request_id': 'req-a75f6c03-e539-4042-a256-830df1776ded'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:42.613932 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a75f6c03-e539-4042-a256-830df1776ded admin admin] The token cache is being invalidate because project b2d2cad74b8442038d729add08410df8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:42.614269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a75f6c03-e539-4042-a256-830df1776ded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:42.615255 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a75f6c03-e539-4042-a256-830df1776ded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:42.618174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1114/2221] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:42 2022] DELETE /identity/v3/projects/b2d2cad74b8442038d729add08410df8 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:42.622689 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f6c03-e539-4042-a256-830df1776ded admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:42.624123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a75f6c03-e539-4042-a256-830df1776ded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:42.649388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f6c03-e539-4042-a256-830df1776ded admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:42.666272 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cf02c-c5a2-4b96-9277-67bf65393928 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:42.668331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f76e0-f2df-455a-9899-db5f6b09a9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:42.670213 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f76e0-f2df-455a-9899-db5f6b09a9bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:42.670867 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f76e0-f2df-455a-9899-db5f6b09a9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:42.671156 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f76e0-f2df-455a-9899-db5f6b09a9bd admin admin] PATH_INFO: `/v3/users/7cc2530d7b774a88aad0f82faf23e2ad` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:42.671892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd30a16-1db6-4ff4-8bb8-364ee32ad379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:42.672742 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd30a16-1db6-4ff4-8bb8-364ee32ad379 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:42.673218 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd30a16-1db6-4ff4-8bb8-364ee32ad379 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:42.673678 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd30a16-1db6-4ff4-8bb8-364ee32ad379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:42.673791 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f76e0-f2df-455a-9899-db5f6b09a9bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cc2530d7b774a88aad0f82faf23e2ad)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:42.676201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-fcd30a16-1db6-4ff4-8bb8-364ee32ad379 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:42.678233 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f76e0-f2df-455a-9899-db5f6b09a9bd admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:42.699656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd30a16-1db6-4ff4-8bb8-364ee32ad379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:42.700986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd30a16-1db6-4ff4-8bb8-364ee32ad379 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:42.720280 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1108/2222] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:18:42.895603 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ef1f76e0-f2df-455a-9899-db5f6b09a9bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cc2530d7b774a88aad0f82faf23e2ad'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:42.896332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ef1f76e0-f2df-455a-9899-db5f6b09a9bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cc2530d7b774a88aad0f82faf23e2ad'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:42.896932 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ef1f76e0-f2df-455a-9899-db5f6b09a9bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cc2530d7b774a88aad0f82faf23e2ad'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:42.909600 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ef1f76e0-f2df-455a-9899-db5f6b09a9bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cc2530d7b774a88aad0f82faf23e2ad'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:42.916928 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ef1f76e0-f2df-455a-9899-db5f6b09a9bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cc2530d7b774a88aad0f82faf23e2ad'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:43.001521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1115/2223] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:42 2022] DELETE /identity/v3/users/7cc2530d7b774a88aad0f82faf23e2ad => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:43.709710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd30a16-1db6-4ff4-8bb8-364ee32ad379 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:43.727493 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd30a16-1db6-4ff4-8bb8-364ee32ad379 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:43.824496 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29550d7-d6a9-4cff-a1c3-9f142afef898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:43.826138 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29550d7-d6a9-4cff-a1c3-9f142afef898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:43.826582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29550d7-d6a9-4cff-a1c3-9f142afef898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:43.826833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29550d7-d6a9-4cff-a1c3-9f142afef898 admin admin] PATH_INFO: `/v3/projects/fabb8f902d1d4fe59fac437216e9b61e` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:43.828393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29550d7-d6a9-4cff-a1c3-9f142afef898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fabb8f902d1d4fe59fac437216e9b61e)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:43.830329 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29550d7-d6a9-4cff-a1c3-9f142afef898 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:43.986175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f29550d7-d6a9-4cff-a1c3-9f142afef898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fabb8f902d1d4fe59fac437216e9b61e', 'request_id': 'req-f29550d7-d6a9-4cff-a1c3-9f142afef898'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:44.054154 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f29550d7-d6a9-4cff-a1c3-9f142afef898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fabb8f902d1d4fe59fac437216e9b61e', 'request_id': 'req-f29550d7-d6a9-4cff-a1c3-9f142afef898'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:44.062003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f29550d7-d6a9-4cff-a1c3-9f142afef898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fabb8f902d1d4fe59fac437216e9b61e', 'request_id': 'req-f29550d7-d6a9-4cff-a1c3-9f142afef898'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:44.075306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f29550d7-d6a9-4cff-a1c3-9f142afef898 admin admin] The token cache is being invalidate because project fabb8f902d1d4fe59fac437216e9b61e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:44.075588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f29550d7-d6a9-4cff-a1c3-9f142afef898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:44.076262 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f29550d7-d6a9-4cff-a1c3-9f142afef898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:44.077862 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1109/2224] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:43 2022] DELETE /identity/v3/projects/fabb8f902d1d4fe59fac437216e9b61e => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:44.083772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa944460-6e68-4a6e-9677-2f6b9e34059d None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:44.084315 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa944460-6e68-4a6e-9677-2f6b9e34059d None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:44.084558 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa944460-6e68-4a6e-9677-2f6b9e34059d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:44.117187 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-fa944460-6e68-4a6e-9677-2f6b9e34059d None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:44.119429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-fa944460-6e68-4a6e-9677-2f6b9e34059d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:44.197480 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1116/2225] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:44.205433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ec216-578c-42eb-bf84-83628bf16caa None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:44.205788 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ec216-578c-42eb-bf84-83628bf16caa None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:44.206012 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ec216-578c-42eb-bf84-83628bf16caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:44.225794 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-085ec216-578c-42eb-bf84-83628bf16caa None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:44.229655 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-085ec216-578c-42eb-bf84-83628bf16caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:44.250492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1110/2226] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:18:44.251513 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa944460-6e68-4a6e-9677-2f6b9e34059d None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:44.271768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa944460-6e68-4a6e-9677-2f6b9e34059d None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:44.292556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9037f8-fd0f-49e6-9e4b-f4cfe3824d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:44.293694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9037f8-fd0f-49e6-9e4b-f4cfe3824d33 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:44.294863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9037f8-fd0f-49e6-9e4b-f4cfe3824d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:44.295303 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9037f8-fd0f-49e6-9e4b-f4cfe3824d33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:44.296373 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9037f8-fd0f-49e6-9e4b-f4cfe3824d33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:18:44.297061 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9037f8-fd0f-49e6-9e4b-f4cfe3824d33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:44.299135 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9037f8-fd0f-49e6-9e4b-f4cfe3824d33 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:44.314727 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1117/2227] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:18:44.319916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ec216-578c-42eb-bf84-83628bf16caa None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:44.331930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ec216-578c-42eb-bf84-83628bf16caa None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:44.337746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06c40e-b188-4b0f-9512-de178ec65489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:44.339139 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06c40e-b188-4b0f-9512-de178ec65489 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:44.339139 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06c40e-b188-4b0f-9512-de178ec65489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:44.339524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06c40e-b188-4b0f-9512-de178ec65489 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:44.340374 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06c40e-b188-4b0f-9512-de178ec65489 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:44.342700 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06c40e-b188-4b0f-9512-de178ec65489 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:44.447005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1111/2228] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:44 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:44.457733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9037f8-fd0f-49e6-9e4b-f4cfe3824d33 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:44.470594 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9037f8-fd0f-49e6-9e4b-f4cfe3824d33 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:44.546172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11738b4d-f41f-4237-b2cc-ef64cc03d888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:44.546971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11738b4d-f41f-4237-b2cc-ef64cc03d888 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:44.547385 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11738b4d-f41f-4237-b2cc-ef64cc03d888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:44.547734 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11738b4d-f41f-4237-b2cc-ef64cc03d888 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:44.548516 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11738b4d-f41f-4237-b2cc-ef64cc03d888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:44.550830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11738b4d-f41f-4237-b2cc-ef64cc03d888 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:44.650455 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1118/2229] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:44 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:44.654577 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06c40e-b188-4b0f-9512-de178ec65489 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:44.680449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06c40e-b188-4b0f-9512-de178ec65489 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:44.695176 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28a00b-93af-4f8c-ae42-781f8adb528a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:44.697471 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28a00b-93af-4f8c-ae42-781f8adb528a admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:44.697471 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28a00b-93af-4f8c-ae42-781f8adb528a admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:44.697771 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28a00b-93af-4f8c-ae42-781f8adb528a admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:44.698582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd28a00b-93af-4f8c-ae42-781f8adb528a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:44.701799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd28a00b-93af-4f8c-ae42-781f8adb528a admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:44.727729 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11738b4d-f41f-4237-b2cc-ef64cc03d888 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:44.741077 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1112/2230] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:44.745718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11738b4d-f41f-4237-b2cc-ef64cc03d888 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:44.752998 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9533a-bad2-4f75-8dba-9477c571dbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:44.762633 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9533a-bad2-4f75-8dba-9477c571dbb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:44.762633 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9533a-bad2-4f75-8dba-9477c571dbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:44.762633 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9533a-bad2-4f75-8dba-9477c571dbb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:44.762633 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9533a-bad2-4f75-8dba-9477c571dbb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:44.762633 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9533a-bad2-4f75-8dba-9477c571dbb8 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:44.773042 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1119/2231] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:44.777633 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28a00b-93af-4f8c-ae42-781f8adb528a admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:44.790541 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28a00b-93af-4f8c-ae42-781f8adb528a admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:44.796622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d59c1-45dd-43d7-b15d-32b9e15f3eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:44.797575 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d59c1-45dd-43d7-b15d-32b9e15f3eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:44.798072 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d59c1-45dd-43d7-b15d-32b9e15f3eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:44.798621 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d59c1-45dd-43d7-b15d-32b9e15f3eb5 admin admin] PATH_INFO: `/v3/projects/1282aa50cd8d4561a311ceab85c794fa/users/9848d44446b54e7c981e6e61d25e4f28/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:44.830780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d59c1-45dd-43d7-b15d-32b9e15f3eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1282aa50cd8d4561a311ceab85c794fa, user_id=9848d44446b54e7c981e6e61d25e4f28, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:44.833463 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d59c1-45dd-43d7-b15d-32b9e15f3eb5 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:44.976563 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1113/2232] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:44 2022] PUT /identity/v3/projects/1282aa50cd8d4561a311ceab85c794fa/users/9848d44446b54e7c981e6e61d25e4f28/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:44.992297 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773309ed-b97d-4280-86b0-9159bd78da10 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:44.994745 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773309ed-b97d-4280-86b0-9159bd78da10 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:45.000225 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773309ed-b97d-4280-86b0-9159bd78da10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:45.110329 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-773309ed-b97d-4280-86b0-9159bd78da10 None None] MFA Rules not processed for user `9848d44446b54e7c981e6e61d25e4f28`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:45.113727 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-773309ed-b97d-4280-86b0-9159bd78da10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:45.259423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1120/2233] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:45.266550 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d59c1-45dd-43d7-b15d-32b9e15f3eb5 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:45.409351 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d59c1-45dd-43d7-b15d-32b9e15f3eb5 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:45.417078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9b4c2-341a-4095-86a6-33b8a69633e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:45.417742 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9b4c2-341a-4095-86a6-33b8a69633e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:45.418033 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9b4c2-341a-4095-86a6-33b8a69633e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:45.418413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9b4c2-341a-4095-86a6-33b8a69633e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:45.419092 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9b4c2-341a-4095-86a6-33b8a69633e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:45.421082 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9b4c2-341a-4095-86a6-33b8a69633e4 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:45.534362 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b9d78-f658-4066-adb6-792fe8cb1ed2 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:45.537478 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b9d78-f658-4066-adb6-792fe8cb1ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:45.537478 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b9d78-f658-4066-adb6-792fe8cb1ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:45.551271 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1114/2234] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:45 2022] POST /identity/v3/projects => generated 377 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:45.552536 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9b4c2-341a-4095-86a6-33b8a69633e4 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:45.599206 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-7b7b9d78-f658-4066-adb6-792fe8cb1ed2 None None] MFA Rules not processed for user `e98e849891bf45768936056e3206d880`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:45.600617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7b7b9d78-f658-4066-adb6-792fe8cb1ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:45.657764 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9b4c2-341a-4095-86a6-33b8a69633e4 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:45.664833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223f572-2738-4003-aaf1-a0d7489f7507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:45.665674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223f572-2738-4003-aaf1-a0d7489f7507 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:45.666052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223f572-2738-4003-aaf1-a0d7489f7507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:45.666447 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223f572-2738-4003-aaf1-a0d7489f7507 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:45.668377 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1223f572-2738-4003-aaf1-a0d7489f7507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:45.670142 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1223f572-2738-4003-aaf1-a0d7489f7507 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:45.700840 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1121/2235] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:45.709515 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b9d78-f658-4066-adb6-792fe8cb1ed2 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:45.711878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7b7b9d78-f658-4066-adb6-792fe8cb1ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:45.776787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1115/2236] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:45 2022] POST /identity/v3/users => generated 289 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:45.781930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223f572-2738-4003-aaf1-a0d7489f7507 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:45.797457 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223f572-2738-4003-aaf1-a0d7489f7507 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:45.805491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26602d-afc2-41b3-8fac-4ecef1194494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:45.806274 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26602d-afc2-41b3-8fac-4ecef1194494 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:45.806686 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b9d78-f658-4066-adb6-792fe8cb1ed2 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:45.806922 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26602d-afc2-41b3-8fac-4ecef1194494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:45.807373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26602d-afc2-41b3-8fac-4ecef1194494 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:45.808404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e26602d-afc2-41b3-8fac-4ecef1194494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:45.810456 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e26602d-afc2-41b3-8fac-4ecef1194494 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:45.811997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025a89a-2d01-4194-b1a2-a5d053eac5d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:45.812733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025a89a-2d01-4194-b1a2-a5d053eac5d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:45.813090 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025a89a-2d01-4194-b1a2-a5d053eac5d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:45.813728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025a89a-2d01-4194-b1a2-a5d053eac5d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:45.824646 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6025a89a-2d01-4194-b1a2-a5d053eac5d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:45.830974 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6025a89a-2d01-4194-b1a2-a5d053eac5d0 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:45.839167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1116/2238] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:45.842173 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1122/2238] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:18:45.843130 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26602d-afc2-41b3-8fac-4ecef1194494 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:45.875444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26602d-afc2-41b3-8fac-4ecef1194494 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:45.894276 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14366f39-50cf-4795-bb2a-812233608c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:45.896481 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14366f39-50cf-4795-bb2a-812233608c32 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:45.896481 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14366f39-50cf-4795-bb2a-812233608c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:45.896481 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14366f39-50cf-4795-bb2a-812233608c32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:45.896481 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14366f39-50cf-4795-bb2a-812233608c32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:45.900150 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14366f39-50cf-4795-bb2a-812233608c32 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:45.943521 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1117/2239] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:18:45.948160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025a89a-2d01-4194-b1a2-a5d053eac5d0 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:45.971636 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025a89a-2d01-4194-b1a2-a5d053eac5d0 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:45.977545 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f779d-2917-4b8b-92af-6ebd3557094e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:45.978485 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f779d-2917-4b8b-92af-6ebd3557094e admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:45.978763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f779d-2917-4b8b-92af-6ebd3557094e admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:45.979043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f779d-2917-4b8b-92af-6ebd3557094e admin admin] PATH_INFO: `/v3/projects/06bae9e36ff54a98acd5355b08ba0b43/users/1ab74916538a47bebe1dd1a3a69a31ec/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:46.021757 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f779d-2917-4b8b-92af-6ebd3557094e admin admin] RBAC: Authorizing `identity:create_grant(project_id=06bae9e36ff54a98acd5355b08ba0b43, user_id=1ab74916538a47bebe1dd1a3a69a31ec, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:46.025054 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f779d-2917-4b8b-92af-6ebd3557094e admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:46.121046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1123/2240] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:45 2022] PUT /identity/v3/projects/06bae9e36ff54a98acd5355b08ba0b43/users/1ab74916538a47bebe1dd1a3a69a31ec/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:46.125116 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ea6fa-901c-4a35-b50b-533bb451f346 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:46.125901 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ea6fa-901c-4a35-b50b-533bb451f346 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:46.126630 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ea6fa-901c-4a35-b50b-533bb451f346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:46.160451 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-cf8ea6fa-901c-4a35-b50b-533bb451f346 None None] MFA Rules not processed for user `1ab74916538a47bebe1dd1a3a69a31ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:46.163730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-cf8ea6fa-901c-4a35-b50b-533bb451f346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:46.277214 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1118/2241] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:18:46.284370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f779d-2917-4b8b-92af-6ebd3557094e admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:46.377904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f779d-2917-4b8b-92af-6ebd3557094e admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:46.385203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334e428-7b80-4fc9-9903-35e686b425f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:46.385966 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334e428-7b80-4fc9-9903-35e686b425f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:46.386377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334e428-7b80-4fc9-9903-35e686b425f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:46.388214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334e428-7b80-4fc9-9903-35e686b425f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:46.388214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334e428-7b80-4fc9-9903-35e686b425f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:46.390290 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334e428-7b80-4fc9-9903-35e686b425f7 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:46.514109 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1124/2242] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:46 2022] POST /identity/v3/projects => generated 379 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:46.516244 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ea6fa-901c-4a35-b50b-533bb451f346 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:46.534366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ea6fa-901c-4a35-b50b-533bb451f346 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:46.640483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a483283-9d96-4451-b4f2-327ffa5ee5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:46.641231 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a483283-9d96-4451-b4f2-327ffa5ee5b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:46.641512 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a483283-9d96-4451-b4f2-327ffa5ee5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:46.657088 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a483283-9d96-4451-b4f2-327ffa5ee5b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:46.659718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a483283-9d96-4451-b4f2-327ffa5ee5b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:46.661690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a483283-9d96-4451-b4f2-327ffa5ee5b0 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:46.827381 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1119/2243] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:46 2022] POST /identity/v3/users => generated 296 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:18:46.837878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334e428-7b80-4fc9-9903-35e686b425f7 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:46.866268 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334e428-7b80-4fc9-9903-35e686b425f7 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:46.895493 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62d16b-2aa8-4605-9a83-62a1e60b72fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:46.897256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62d16b-2aa8-4605-9a83-62a1e60b72fc admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:46.897256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62d16b-2aa8-4605-9a83-62a1e60b72fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:46.897613 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62d16b-2aa8-4605-9a83-62a1e60b72fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:46.900582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62d16b-2aa8-4605-9a83-62a1e60b72fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:46.903435 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62d16b-2aa8-4605-9a83-62a1e60b72fc admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:46.941854 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1125/2244] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:46.946820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a483283-9d96-4451-b4f2-327ffa5ee5b0 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:46.960355 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a483283-9d96-4451-b4f2-327ffa5ee5b0 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:46.977885 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff27d8b-0d07-4742-be5d-2eeb28006176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:46.979138 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff27d8b-0d07-4742-be5d-2eeb28006176 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:46.979567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff27d8b-0d07-4742-be5d-2eeb28006176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:46.979984 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff27d8b-0d07-4742-be5d-2eeb28006176 admin admin] PATH_INFO: `/v3/projects/fcd1c542a67b4fe4870fc9d1b6750bde/users/22a56d5656e8434bbbc2b031d68f756b/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:47.029575 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff27d8b-0d07-4742-be5d-2eeb28006176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcd1c542a67b4fe4870fc9d1b6750bde, user_id=22a56d5656e8434bbbc2b031d68f756b, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:47.032650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff27d8b-0d07-4742-be5d-2eeb28006176 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:47.722509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1120/2245] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:46 2022] PUT /identity/v3/projects/fcd1c542a67b4fe4870fc9d1b6750bde/users/22a56d5656e8434bbbc2b031d68f756b/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 777 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:47.726321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62d16b-2aa8-4605-9a83-62a1e60b72fc admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:47.738273 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62d16b-2aa8-4605-9a83-62a1e60b72fc admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:47.855423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bfd0a-c96e-4077-95d3-8b6c572f76cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:47.855958 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bfd0a-c96e-4077-95d3-8b6c572f76cb admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:47.856230 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bfd0a-c96e-4077-95d3-8b6c572f76cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:47.856511 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bfd0a-c96e-4077-95d3-8b6c572f76cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:47.857764 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bfd0a-c96e-4077-95d3-8b6c572f76cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:47.858955 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bfd0a-c96e-4077-95d3-8b6c572f76cb admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:47.870425 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1126/2246] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:47.875082 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff27d8b-0d07-4742-be5d-2eeb28006176 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:47.888419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff27d8b-0d07-4742-be5d-2eeb28006176 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:47.894632 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178b6ac-a846-4a77-9f48-b652b5aab2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:47.895526 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178b6ac-a846-4a77-9f48-b652b5aab2a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:47.896026 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178b6ac-a846-4a77-9f48-b652b5aab2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:47.896442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178b6ac-a846-4a77-9f48-b652b5aab2a1 admin admin] PATH_INFO: `/v3/projects/fcd1c542a67b4fe4870fc9d1b6750bde/users/22a56d5656e8434bbbc2b031d68f756b/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:47.898835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178b6ac-a846-4a77-9f48-b652b5aab2a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcd1c542a67b4fe4870fc9d1b6750bde, user_id=22a56d5656e8434bbbc2b031d68f756b, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:47.901786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178b6ac-a846-4a77-9f48-b652b5aab2a1 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:47.913083 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1121/2247] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:47 2022] PUT /identity/v3/projects/fcd1c542a67b4fe4870fc9d1b6750bde/users/22a56d5656e8434bbbc2b031d68f756b/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:47.923172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bfd0a-c96e-4077-95d3-8b6c572f76cb admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:47.923172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-1a8bfd0a-c96e-4077-95d3-8b6c572f76cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:47.926453 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddee2dd-cb3c-4076-a57a-b75e89596013 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:47.926943 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddee2dd-cb3c-4076-a57a-b75e89596013 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:47.927388 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddee2dd-cb3c-4076-a57a-b75e89596013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:47.941686 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bfd0a-c96e-4077-95d3-8b6c572f76cb admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:47.961024 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-0ddee2dd-cb3c-4076-a57a-b75e89596013 None None] MFA Rules not processed for user `22a56d5656e8434bbbc2b031d68f756b`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:47.962680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0ddee2dd-cb3c-4076-a57a-b75e89596013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:48.035727 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7574d0b-7d6b-4f45-8b56-941f78ca794d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:48.041736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7574d0b-7d6b-4f45-8b56-941f78ca794d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:48.041736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7574d0b-7d6b-4f45-8b56-941f78ca794d admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:48.041736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7574d0b-7d6b-4f45-8b56-941f78ca794d admin admin] PATH_INFO: `/v3/users/2ffb8c46abaa474483a2c4e3926bd1fa` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:48.041736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7574d0b-7d6b-4f45-8b56-941f78ca794d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ffb8c46abaa474483a2c4e3926bd1fa)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:48.041736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7574d0b-7d6b-4f45-8b56-941f78ca794d admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:48.090837 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1122/2248] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:18:48.098642 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddee2dd-cb3c-4076-a57a-b75e89596013 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:48.176695 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddee2dd-cb3c-4076-a57a-b75e89596013 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:48.180477 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31905c6c-d32d-4567-8a93-fc66adc4f3e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:48.181250 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31905c6c-d32d-4567-8a93-fc66adc4f3e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:48.181458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31905c6c-d32d-4567-8a93-fc66adc4f3e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:48.181736 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31905c6c-d32d-4567-8a93-fc66adc4f3e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:48.193432 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31905c6c-d32d-4567-8a93-fc66adc4f3e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:48.194359 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31905c6c-d32d-4567-8a93-fc66adc4f3e6 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:48.206360 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1123/2249] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:48.330121 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b7574d0b-7d6b-4f45-8b56-941f78ca794d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ffb8c46abaa474483a2c4e3926bd1fa'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:48.331283 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b7574d0b-7d6b-4f45-8b56-941f78ca794d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ffb8c46abaa474483a2c4e3926bd1fa'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:48.331991 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b7574d0b-7d6b-4f45-8b56-941f78ca794d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ffb8c46abaa474483a2c4e3926bd1fa'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:48.351390 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31905c6c-d32d-4567-8a93-fc66adc4f3e6 service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:48.357262 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-31905c6c-d32d-4567-8a93-fc66adc4f3e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:48.359831 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b7574d0b-7d6b-4f45-8b56-941f78ca794d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ffb8c46abaa474483a2c4e3926bd1fa'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:48.372241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-b7574d0b-7d6b-4f45-8b56-941f78ca794d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ffb8c46abaa474483a2c4e3926bd1fa'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:48.375027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31905c6c-d32d-4567-8a93-fc66adc4f3e6 service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:48.390614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e8882-e3da-464e-95c1-889c367b5ab0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:48.392108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e8882-e3da-464e-95c1-889c367b5ab0 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:48.393798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e8882-e3da-464e-95c1-889c367b5ab0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:48.395009 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e8882-e3da-464e-95c1-889c367b5ab0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:48.398220 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-286e8882-e3da-464e-95c1-889c367b5ab0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:48.476427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1127/2250] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:47 2022] DELETE /identity/v3/users/2ffb8c46abaa474483a2c4e3926bd1fa => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:48.482988 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d9b9a-5f43-4b5e-96f0-f790749d566c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:48.483345 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d9b9a-5f43-4b5e-96f0-f790749d566c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:48.483537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d9b9a-5f43-4b5e-96f0-f790749d566c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:48.534776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-db5d9b9a-5f43-4b5e-96f0-f790749d566c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:48.538482 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e8882-e3da-464e-95c1-889c367b5ab0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:48.539062 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-db5d9b9a-5f43-4b5e-96f0-f790749d566c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:48.541035 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e8882-e3da-464e-95c1-889c367b5ab0 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:48.567086 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1124/2251] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:48.627296 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1128/2252] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:48.650452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e8882-e3da-464e-95c1-889c367b5ab0 service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:48.650452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-286e8882-e3da-464e-95c1-889c367b5ab0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:48.715285 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e8882-e3da-464e-95c1-889c367b5ab0 service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:48.780463 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3181368-ee85-4c87-992b-f4e09f90d047 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:48.781470 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3181368-ee85-4c87-992b-f4e09f90d047 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:48.781470 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3181368-ee85-4c87-992b-f4e09f90d047 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:48.781857 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3181368-ee85-4c87-992b-f4e09f90d047 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:48.794380 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3181368-ee85-4c87-992b-f4e09f90d047 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:48.795917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3181368-ee85-4c87-992b-f4e09f90d047 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:48.808086 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1125/2253] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:48.993671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93798f7b-8975-432e-8168-d2ab584d0546 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:48.993671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93798f7b-8975-432e-8168-d2ab584d0546 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:48.993671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93798f7b-8975-432e-8168-d2ab584d0546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:49.014606 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-93798f7b-8975-432e-8168-d2ab584d0546 None None] MFA Rules not processed for user `e98e849891bf45768936056e3206d880`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:49.016184 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-93798f7b-8975-432e-8168-d2ab584d0546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:49.126954 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1129/2254] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:49.655079 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3181368-ee85-4c87-992b-f4e09f90d047 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:49.658882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f3181368-ee85-4c87-992b-f4e09f90d047 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:49.680368 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3181368-ee85-4c87-992b-f4e09f90d047 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:49.692152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f845ed-a24e-4559-a9bc-775c1b94d86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:49.693394 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f845ed-a24e-4559-a9bc-775c1b94d86e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:49.693892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f845ed-a24e-4559-a9bc-775c1b94d86e admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:49.694373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f845ed-a24e-4559-a9bc-775c1b94d86e admin admin] PATH_INFO: `/v3/projects/c3d8d2fe430e47a3bdca084e051efc72` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:49.698463 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f845ed-a24e-4559-a9bc-775c1b94d86e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d8d2fe430e47a3bdca084e051efc72)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:49.703099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f845ed-a24e-4559-a9bc-775c1b94d86e admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:49.886971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-65f845ed-a24e-4559-a9bc-775c1b94d86e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d8d2fe430e47a3bdca084e051efc72', 'request_id': 'req-65f845ed-a24e-4559-a9bc-775c1b94d86e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:49.980913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-65f845ed-a24e-4559-a9bc-775c1b94d86e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d8d2fe430e47a3bdca084e051efc72', 'request_id': 'req-65f845ed-a24e-4559-a9bc-775c1b94d86e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:49.988344 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-65f845ed-a24e-4559-a9bc-775c1b94d86e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d8d2fe430e47a3bdca084e051efc72', 'request_id': 'req-65f845ed-a24e-4559-a9bc-775c1b94d86e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:50.000955 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-65f845ed-a24e-4559-a9bc-775c1b94d86e admin admin] The token cache is being invalidate because project c3d8d2fe430e47a3bdca084e051efc72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:50.001352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-65f845ed-a24e-4559-a9bc-775c1b94d86e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:50.002127 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-65f845ed-a24e-4559-a9bc-775c1b94d86e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:50.004081 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1126/2255] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:49 2022] DELETE /identity/v3/projects/c3d8d2fe430e47a3bdca084e051efc72 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:50.009459 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0e9e5-c9f7-4647-82c3-e760bb70ad4e None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:50.010497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0e9e5-c9f7-4647-82c3-e760bb70ad4e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:50.011027 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0e9e5-c9f7-4647-82c3-e760bb70ad4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:50.031776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-74a0e9e5-c9f7-4647-82c3-e760bb70ad4e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:50.033254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-74a0e9e5-c9f7-4647-82c3-e760bb70ad4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:50.155060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1130/2256] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:18:50.163433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617e6a6-2ef4-49c7-af1c-8f74605dc150 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:50.163729 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617e6a6-2ef4-49c7-af1c-8f74605dc150 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:50.164094 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617e6a6-2ef4-49c7-af1c-8f74605dc150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:50.187803 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-f617e6a6-2ef4-49c7-af1c-8f74605dc150 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:50.188341 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f617e6a6-2ef4-49c7-af1c-8f74605dc150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:50.199945 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1127/2257] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:50.204947 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0e9e5-c9f7-4647-82c3-e760bb70ad4e None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:50.224177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0e9e5-c9f7-4647-82c3-e760bb70ad4e None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:50.230643 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d3fa3-8075-47f8-ae3d-6b3ea26a4116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:50.231531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d3fa3-8075-47f8-ae3d-6b3ea26a4116 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:50.231990 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d3fa3-8075-47f8-ae3d-6b3ea26a4116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:50.232419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d3fa3-8075-47f8-ae3d-6b3ea26a4116 admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:50.234327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d3fa3-8075-47f8-ae3d-6b3ea26a4116 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:18:50.235089 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d3fa3-8075-47f8-ae3d-6b3ea26a4116 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:50.236846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d3fa3-8075-47f8-ae3d-6b3ea26a4116 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:50.265589 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1131/2258] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:18:50.270128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617e6a6-2ef4-49c7-af1c-8f74605dc150 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:50.288492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617e6a6-2ef4-49c7-af1c-8f74605dc150 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:50.294593 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84876d1e-ced6-4c87-976f-3c435ee1e34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:50.295510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84876d1e-ced6-4c87-976f-3c435ee1e34e admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:50.296005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84876d1e-ced6-4c87-976f-3c435ee1e34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:50.296503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84876d1e-ced6-4c87-976f-3c435ee1e34e admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:50.297600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84876d1e-ced6-4c87-976f-3c435ee1e34e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:50.300616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84876d1e-ced6-4c87-976f-3c435ee1e34e admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:50.406317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1128/2259] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:50 2022] POST /identity/v3/projects => generated 389 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:50.410735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d3fa3-8075-47f8-ae3d-6b3ea26a4116 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:50.425702 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d3fa3-8075-47f8-ae3d-6b3ea26a4116 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:50.512566 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe67ff-68f3-40b0-bbc0-2124c88b7647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:50.513753 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe67ff-68f3-40b0-bbc0-2124c88b7647 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:50.514188 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe67ff-68f3-40b0-bbc0-2124c88b7647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:50.514479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe67ff-68f3-40b0-bbc0-2124c88b7647 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:50.516089 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe67ff-68f3-40b0-bbc0-2124c88b7647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:50.518211 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe67ff-68f3-40b0-bbc0-2124c88b7647 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:50.627645 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1132/2260] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:50 2022] POST /identity/v3/users => generated 295 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:18:50.631679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84876d1e-ced6-4c87-976f-3c435ee1e34e admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:50.651002 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84876d1e-ced6-4c87-976f-3c435ee1e34e admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:50.666184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99521a-0ef6-4539-8923-a3bf1e525404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:50.667126 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99521a-0ef6-4539-8923-a3bf1e525404 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:50.667415 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99521a-0ef6-4539-8923-a3bf1e525404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:50.667954 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99521a-0ef6-4539-8923-a3bf1e525404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:50.692896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99521a-0ef6-4539-8923-a3bf1e525404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:50.695328 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99521a-0ef6-4539-8923-a3bf1e525404 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:50.727741 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1129/2261] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:50.730661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe67ff-68f3-40b0-bbc0-2124c88b7647 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:50.747550 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe67ff-68f3-40b0-bbc0-2124c88b7647 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:50.754591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb45e5-332d-46ca-8b4a-a9b51a6c32c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:50.755355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb45e5-332d-46ca-8b4a-a9b51a6c32c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:50.755707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb45e5-332d-46ca-8b4a-a9b51a6c32c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:50.756102 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb45e5-332d-46ca-8b4a-a9b51a6c32c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:50.757068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb45e5-332d-46ca-8b4a-a9b51a6c32c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:50.759630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb45e5-332d-46ca-8b4a-a9b51a6c32c1 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:50.775077 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1133/2262] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:50.776427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99521a-0ef6-4539-8923-a3bf1e525404 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:50.792841 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99521a-0ef6-4539-8923-a3bf1e525404 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:50.798195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e57c5-72bd-4207-ab83-d4722a3c1342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:50.799266 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e57c5-72bd-4207-ab83-d4722a3c1342 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:50.799627 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e57c5-72bd-4207-ab83-d4722a3c1342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:50.799895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e57c5-72bd-4207-ab83-d4722a3c1342 admin admin] PATH_INFO: `/v3/projects/c2077eff313d40b195705c10603825a8/users/b5e37bdf95e04a5f8abc2c6dda2683ea/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:50.815864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717e57c5-72bd-4207-ab83-d4722a3c1342 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2077eff313d40b195705c10603825a8, user_id=b5e37bdf95e04a5f8abc2c6dda2683ea, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:50.819813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717e57c5-72bd-4207-ab83-d4722a3c1342 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:50.954941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1130/2263] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:50 2022] PUT /identity/v3/projects/c2077eff313d40b195705c10603825a8/users/b5e37bdf95e04a5f8abc2c6dda2683ea/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:50.957164 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac2101-6959-40b1-98fb-fe22ed5d30e0 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:50.957724 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac2101-6959-40b1-98fb-fe22ed5d30e0 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:50.958404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac2101-6959-40b1-98fb-fe22ed5d30e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:51.019761 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-8eac2101-6959-40b1-98fb-fe22ed5d30e0 None None] MFA Rules not processed for user `b5e37bdf95e04a5f8abc2c6dda2683ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:51.021114 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-8eac2101-6959-40b1-98fb-fe22ed5d30e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:51.197930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1134/2264] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:51.215964 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e57c5-72bd-4207-ab83-d4722a3c1342 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:51.218248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-717e57c5-72bd-4207-ab83-d4722a3c1342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:51.312670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e57c5-72bd-4207-ab83-d4722a3c1342 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:51.324248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ec5cb-27c8-4b21-9f77-68e2657ed4a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:51.324248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ec5cb-27c8-4b21-9f77-68e2657ed4a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:51.324248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ec5cb-27c8-4b21-9f77-68e2657ed4a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:51.324527 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ec5cb-27c8-4b21-9f77-68e2657ed4a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:51.333582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ec5cb-27c8-4b21-9f77-68e2657ed4a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:51.335317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ec5cb-27c8-4b21-9f77-68e2657ed4a5 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:51.351927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1131/2265] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:53.380650 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c54bf-f51b-47d4-bfc7-996f6a7f7059 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:53.385738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c54bf-f51b-47d4-bfc7-996f6a7f7059 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:53.385738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c54bf-f51b-47d4-bfc7-996f6a7f7059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:53.429555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-b24c54bf-f51b-47d4-bfc7-996f6a7f7059 None None] MFA Rules not processed for user `f213f79341ab4462b3d51352ccb9de2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:53.431075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b24c54bf-f51b-47d4-bfc7-996f6a7f7059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:53.511103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1135/2266] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:53.515829 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ec5cb-27c8-4b21-9f77-68e2657ed4a5 service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:53.517666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-e21ec5cb-27c8-4b21-9f77-68e2657ed4a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:53.592583 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ec5cb-27c8-4b21-9f77-68e2657ed4a5 service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:53.597931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ae6f2-ab81-4c3d-a64d-1134691fc52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:53.598647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ae6f2-ab81-4c3d-a64d-1134691fc52b admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:53.599300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ae6f2-ab81-4c3d-a64d-1134691fc52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:53.599618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ae6f2-ab81-4c3d-a64d-1134691fc52b admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:53.600338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ae6f2-ab81-4c3d-a64d-1134691fc52b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:53.603281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ae6f2-ab81-4c3d-a64d-1134691fc52b admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:53.726707 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1132/2267] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:53 2022] POST /identity/v3/projects => generated 371 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:18:53.731026 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c54bf-f51b-47d4-bfc7-996f6a7f7059 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:53.742573 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c54bf-f51b-47d4-bfc7-996f6a7f7059 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:53.844093 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d258a2-9209-4ae7-8a0a-91df2a842246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:53.844770 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d258a2-9209-4ae7-8a0a-91df2a842246 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:53.845056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d258a2-9209-4ae7-8a0a-91df2a842246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:53.845332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d258a2-9209-4ae7-8a0a-91df2a842246 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:53.846039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d258a2-9209-4ae7-8a0a-91df2a842246 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:53.848043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d258a2-9209-4ae7-8a0a-91df2a842246 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:53.922461 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ae6f2-ab81-4c3d-a64d-1134691fc52b admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:53.924581 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-aa2ae6f2-ab81-4c3d-a64d-1134691fc52b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:53.940286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ae6f2-ab81-4c3d-a64d-1134691fc52b admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:53.949624 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a64e1-fe64-4760-a549-bd13f24a18ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:53.950777 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a64e1-fe64-4760-a549-bd13f24a18ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:53.951994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a64e1-fe64-4760-a549-bd13f24a18ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:53.952224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a64e1-fe64-4760-a549-bd13f24a18ce admin admin] PATH_INFO: `/v3/users/9848d44446b54e7c981e6e61d25e4f28` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:53.953866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a64e1-fe64-4760-a549-bd13f24a18ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=9848d44446b54e7c981e6e61d25e4f28)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:53.956910 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a64e1-fe64-4760-a549-bd13f24a18ce admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:54.002192 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1136/2268] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:18:53 2022] POST /identity/v3/users => generated 292 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:18:54.003329 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d258a2-9209-4ae7-8a0a-91df2a842246 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:54.023716 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d258a2-9209-4ae7-8a0a-91df2a842246 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:54.042000 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829fa70-6c7f-47c7-b8e9-6205955a0709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:54.043086 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829fa70-6c7f-47c7-b8e9-6205955a0709 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:54.043738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829fa70-6c7f-47c7-b8e9-6205955a0709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:54.044152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829fa70-6c7f-47c7-b8e9-6205955a0709 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:54.046670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9829fa70-6c7f-47c7-b8e9-6205955a0709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:54.051307 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9829fa70-6c7f-47c7-b8e9-6205955a0709 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:54.067488 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1137/2269] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:18:54.073513 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829fa70-6c7f-47c7-b8e9-6205955a0709 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:54.084625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829fa70-6c7f-47c7-b8e9-6205955a0709 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:54.094684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2066d-14ba-4be8-b4af-6c946955d7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:54.100947 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2066d-14ba-4be8-b4af-6c946955d7c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:54.100947 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2066d-14ba-4be8-b4af-6c946955d7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:54.100947 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2066d-14ba-4be8-b4af-6c946955d7c0 admin admin] PATH_INFO: `/v3/projects/ef25570884a043a9811f92c3391ec916/users/c91839388e054c0cbe7c8e7bbb35c240/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:54.117419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2066d-14ba-4be8-b4af-6c946955d7c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef25570884a043a9811f92c3391ec916, user_id=c91839388e054c0cbe7c8e7bbb35c240, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:54.122691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2066d-14ba-4be8-b4af-6c946955d7c0 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:54.222185 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1138/2270] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:54 2022] PUT /identity/v3/projects/ef25570884a043a9811f92c3391ec916/users/c91839388e054c0cbe7c8e7bbb35c240/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:54.223965 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2066d-14ba-4be8-b4af-6c946955d7c0 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:54.228258 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-566a64e1-fe64-4760-a549-bd13f24a18ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9848d44446b54e7c981e6e61d25e4f28'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:54.229206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-566a64e1-fe64-4760-a549-bd13f24a18ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9848d44446b54e7c981e6e61d25e4f28'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:54.246242 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-566a64e1-fe64-4760-a549-bd13f24a18ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9848d44446b54e7c981e6e61d25e4f28'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:54.248016 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-566a64e1-fe64-4760-a549-bd13f24a18ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9848d44446b54e7c981e6e61d25e4f28'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:54.259733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-566a64e1-fe64-4760-a549-bd13f24a18ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9848d44446b54e7c981e6e61d25e4f28'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:54.310321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2066d-14ba-4be8-b4af-6c946955d7c0 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:54.315989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6383e61-5d44-438e-8d50-9cad0cfb3042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:54.316919 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6383e61-5d44-438e-8d50-9cad0cfb3042 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:54.317355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6383e61-5d44-438e-8d50-9cad0cfb3042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:54.317792 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6383e61-5d44-438e-8d50-9cad0cfb3042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:54.318566 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6383e61-5d44-438e-8d50-9cad0cfb3042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:54.320380 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6383e61-5d44-438e-8d50-9cad0cfb3042 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:54.332379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1139/2271] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:18:54.336156 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6383e61-5d44-438e-8d50-9cad0cfb3042 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:54.339906 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a6383e61-5d44-438e-8d50-9cad0cfb3042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:54.356908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6383e61-5d44-438e-8d50-9cad0cfb3042 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:54.362239 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b684b-6af1-4901-9a1a-07ac722a171d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:54.363205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b684b-6af1-4901-9a1a-07ac722a171d admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:54.363533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b684b-6af1-4901-9a1a-07ac722a171d admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:54.363774 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b684b-6af1-4901-9a1a-07ac722a171d admin admin] PATH_INFO: `/v3/projects/ef25570884a043a9811f92c3391ec916/users/c91839388e054c0cbe7c8e7bbb35c240/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:54.365779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b684b-6af1-4901-9a1a-07ac722a171d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef25570884a043a9811f92c3391ec916, user_id=c91839388e054c0cbe7c8e7bbb35c240, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:54.369067 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b684b-6af1-4901-9a1a-07ac722a171d admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:54.380951 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1140/2272] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:18:54 2022] PUT /identity/v3/projects/ef25570884a043a9811f92c3391ec916/users/c91839388e054c0cbe7c8e7bbb35c240/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:54.415896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1133/2273] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:53 2022] DELETE /identity/v3/users/9848d44446b54e7c981e6e61d25e4f28 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:54.426117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96960323-16a0-4284-8700-beb777c9bdba None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:54.426117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96960323-16a0-4284-8700-beb777c9bdba None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:54.426335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96960323-16a0-4284-8700-beb777c9bdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:54.449852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-96960323-16a0-4284-8700-beb777c9bdba None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:54.451430 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-96960323-16a0-4284-8700-beb777c9bdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:54.547054 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1141/2274] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:54.566975 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a64e1-fe64-4760-a549-bd13f24a18ce admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:54.568403 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-566a64e1-fe64-4760-a549-bd13f24a18ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:54.695244 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a64e1-fe64-4760-a549-bd13f24a18ce admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:54.708625 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0135ad6f-27a0-4f7d-865c-62d9df5c5b9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:54.709299 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135ad6f-27a0-4f7d-865c-62d9df5c5b9b service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:54.709758 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135ad6f-27a0-4f7d-865c-62d9df5c5b9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:54.710042 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135ad6f-27a0-4f7d-865c-62d9df5c5b9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:54.726963 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0135ad6f-27a0-4f7d-865c-62d9df5c5b9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:54.729003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0135ad6f-27a0-4f7d-865c-62d9df5c5b9b service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:54.758464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1134/2275] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:18:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:18:55.533438 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96960323-16a0-4284-8700-beb777c9bdba None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:55.535696 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-96960323-16a0-4284-8700-beb777c9bdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:55.552331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96960323-16a0-4284-8700-beb777c9bdba None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:55.565087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e42649-870e-4233-b6b9-3b60ff272800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:55.566696 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e42649-870e-4233-b6b9-3b60ff272800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:55.567168 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e42649-870e-4233-b6b9-3b60ff272800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:55.567748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e42649-870e-4233-b6b9-3b60ff272800 admin admin] PATH_INFO: `/v3/projects/1282aa50cd8d4561a311ceab85c794fa` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:55.569220 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e42649-870e-4233-b6b9-3b60ff272800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1282aa50cd8d4561a311ceab85c794fa)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:55.571874 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e42649-870e-4233-b6b9-3b60ff272800 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:55.692788 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-30e42649-870e-4233-b6b9-3b60ff272800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1282aa50cd8d4561a311ceab85c794fa', 'request_id': 'req-30e42649-870e-4233-b6b9-3b60ff272800'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:55.829638 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-30e42649-870e-4233-b6b9-3b60ff272800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1282aa50cd8d4561a311ceab85c794fa', 'request_id': 'req-30e42649-870e-4233-b6b9-3b60ff272800'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:55.836669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-30e42649-870e-4233-b6b9-3b60ff272800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1282aa50cd8d4561a311ceab85c794fa', 'request_id': 'req-30e42649-870e-4233-b6b9-3b60ff272800'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:55.849865 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-30e42649-870e-4233-b6b9-3b60ff272800 admin admin] The token cache is being invalidate because project 1282aa50cd8d4561a311ceab85c794fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:55.850286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-30e42649-870e-4233-b6b9-3b60ff272800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:55.851446 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-30e42649-870e-4233-b6b9-3b60ff272800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:55.856770 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1142/2276] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:55 2022] DELETE /identity/v3/projects/1282aa50cd8d4561a311ceab85c794fa => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:55.863340 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0135ad6f-27a0-4f7d-865c-62d9df5c5b9b service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:55.871878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0135ad6f-27a0-4f7d-865c-62d9df5c5b9b service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:55.973732 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c0639-4a54-44ed-9858-29abb4c2fa37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:55.975783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c0639-4a54-44ed-9858-29abb4c2fa37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:55.976093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c0639-4a54-44ed-9858-29abb4c2fa37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:55.976378 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c0639-4a54-44ed-9858-29abb4c2fa37 admin admin] PATH_INFO: `/v3/users/1ab74916538a47bebe1dd1a3a69a31ec` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:55.980484 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c0639-4a54-44ed-9858-29abb4c2fa37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ab74916538a47bebe1dd1a3a69a31ec)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:55.983104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c0639-4a54-44ed-9858-29abb4c2fa37 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:56.200464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-850c0639-4a54-44ed-9858-29abb4c2fa37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ab74916538a47bebe1dd1a3a69a31ec'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:56.201472 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-850c0639-4a54-44ed-9858-29abb4c2fa37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ab74916538a47bebe1dd1a3a69a31ec'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:56.215313 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-850c0639-4a54-44ed-9858-29abb4c2fa37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ab74916538a47bebe1dd1a3a69a31ec'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:56.216326 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-850c0639-4a54-44ed-9858-29abb4c2fa37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ab74916538a47bebe1dd1a3a69a31ec'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:56.226932 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-850c0639-4a54-44ed-9858-29abb4c2fa37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ab74916538a47bebe1dd1a3a69a31ec'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:56.315648 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1135/2277] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:55 2022] DELETE /identity/v3/users/1ab74916538a47bebe1dd1a3a69a31ec => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:18:57.596999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e42649-870e-4233-b6b9-3b60ff272800 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:57.603898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-30e42649-870e-4233-b6b9-3b60ff272800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:57.621067 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e42649-870e-4233-b6b9-3b60ff272800 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:57.761245 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f1e0a-d9d0-4bdd-8e39-bb3e5a70118f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:57.762176 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f1e0a-d9d0-4bdd-8e39-bb3e5a70118f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:57.763510 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f1e0a-d9d0-4bdd-8e39-bb3e5a70118f admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:57.763954 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f1e0a-d9d0-4bdd-8e39-bb3e5a70118f admin admin] PATH_INFO: `/v3/projects/06bae9e36ff54a98acd5355b08ba0b43` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:57.766937 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f1e0a-d9d0-4bdd-8e39-bb3e5a70118f admin admin] RBAC: Authorizing `identity:delete_project(project_id=06bae9e36ff54a98acd5355b08ba0b43)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:57.768980 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f1e0a-d9d0-4bdd-8e39-bb3e5a70118f admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:57.899175 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-529f1e0a-d9d0-4bdd-8e39-bb3e5a70118f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06bae9e36ff54a98acd5355b08ba0b43', 'request_id': 'req-529f1e0a-d9d0-4bdd-8e39-bb3e5a70118f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:57.997740 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-529f1e0a-d9d0-4bdd-8e39-bb3e5a70118f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06bae9e36ff54a98acd5355b08ba0b43', 'request_id': 'req-529f1e0a-d9d0-4bdd-8e39-bb3e5a70118f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:58.014541 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-529f1e0a-d9d0-4bdd-8e39-bb3e5a70118f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06bae9e36ff54a98acd5355b08ba0b43', 'request_id': 'req-529f1e0a-d9d0-4bdd-8e39-bb3e5a70118f'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:58.031406 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-529f1e0a-d9d0-4bdd-8e39-bb3e5a70118f admin admin] The token cache is being invalidate because project 06bae9e36ff54a98acd5355b08ba0b43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:58.031950 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-529f1e0a-d9d0-4bdd-8e39-bb3e5a70118f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:58.033141 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-529f1e0a-d9d0-4bdd-8e39-bb3e5a70118f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:58.039138 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1143/2278] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:57 2022] DELETE /identity/v3/projects/06bae9e36ff54a98acd5355b08ba0b43 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:58.039859 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c0639-4a54-44ed-9858-29abb4c2fa37 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:58.041945 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-850c0639-4a54-44ed-9858-29abb4c2fa37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:58.061872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c0639-4a54-44ed-9858-29abb4c2fa37 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:58.188255 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633461ac-7ba5-4c27-acc2-3d60fe4991b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:58.189265 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633461ac-7ba5-4c27-acc2-3d60fe4991b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:58.189878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633461ac-7ba5-4c27-acc2-3d60fe4991b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:58.190252 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633461ac-7ba5-4c27-acc2-3d60fe4991b3 admin admin] PATH_INFO: `/v3/users/22a56d5656e8434bbbc2b031d68f756b` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:58.193367 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633461ac-7ba5-4c27-acc2-3d60fe4991b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22a56d5656e8434bbbc2b031d68f756b)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:58.194686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633461ac-7ba5-4c27-acc2-3d60fe4991b3 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:58.461915 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-633461ac-7ba5-4c27-acc2-3d60fe4991b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22a56d5656e8434bbbc2b031d68f756b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:58.462671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-633461ac-7ba5-4c27-acc2-3d60fe4991b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22a56d5656e8434bbbc2b031d68f756b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:58.488321 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-633461ac-7ba5-4c27-acc2-3d60fe4991b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22a56d5656e8434bbbc2b031d68f756b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:58.489085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-633461ac-7ba5-4c27-acc2-3d60fe4991b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22a56d5656e8434bbbc2b031d68f756b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:58.516848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-633461ac-7ba5-4c27-acc2-3d60fe4991b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22a56d5656e8434bbbc2b031d68f756b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:58.604724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1136/2279] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:18:58 2022] DELETE /identity/v3/users/22a56d5656e8434bbbc2b031d68f756b => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:59.327636 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f1e0a-d9d0-4bdd-8e39-bb3e5a70118f admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:59.329405 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-529f1e0a-d9d0-4bdd-8e39-bb3e5a70118f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:59.350801 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f1e0a-d9d0-4bdd-8e39-bb3e5a70118f admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:59.440798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b890ab0-317d-429b-8eeb-a02c53c9ebd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:59.442410 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b890ab0-317d-429b-8eeb-a02c53c9ebd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:59.442704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b890ab0-317d-429b-8eeb-a02c53c9ebd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:59.443334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b890ab0-317d-429b-8eeb-a02c53c9ebd9 admin admin] PATH_INFO: `/v3/projects/fcd1c542a67b4fe4870fc9d1b6750bde` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:59.444665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b890ab0-317d-429b-8eeb-a02c53c9ebd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcd1c542a67b4fe4870fc9d1b6750bde)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:59.446846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b890ab0-317d-429b-8eeb-a02c53c9ebd9 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:59.585503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8b890ab0-317d-429b-8eeb-a02c53c9ebd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcd1c542a67b4fe4870fc9d1b6750bde', 'request_id': 'req-8b890ab0-317d-429b-8eeb-a02c53c9ebd9'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:59.667125 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8b890ab0-317d-429b-8eeb-a02c53c9ebd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcd1c542a67b4fe4870fc9d1b6750bde', 'request_id': 'req-8b890ab0-317d-429b-8eeb-a02c53c9ebd9'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:59.673384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8b890ab0-317d-429b-8eeb-a02c53c9ebd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcd1c542a67b4fe4870fc9d1b6750bde', 'request_id': 'req-8b890ab0-317d-429b-8eeb-a02c53c9ebd9'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:59.685025 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8b890ab0-317d-429b-8eeb-a02c53c9ebd9 admin admin] The token cache is being invalidate because project fcd1c542a67b4fe4870fc9d1b6750bde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:18:59.685313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8b890ab0-317d-429b-8eeb-a02c53c9ebd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:59.686113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8b890ab0-317d-429b-8eeb-a02c53c9ebd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:18:59.687744 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1144/2280] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:18:59 2022] DELETE /identity/v3/projects/fcd1c542a67b4fe4870fc9d1b6750bde => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:18:59.694047 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02295d42-88f3-4b85-9c92-aec569c9faa4 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:59.694911 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02295d42-88f3-4b85-9c92-aec569c9faa4 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:59.695355 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02295d42-88f3-4b85-9c92-aec569c9faa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:59.716506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-02295d42-88f3-4b85-9c92-aec569c9faa4 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:59.717871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-02295d42-88f3-4b85-9c92-aec569c9faa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:59.795094 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1137/2281] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:18:59.803928 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae6d60-13e6-442e-979d-608fcc2102fc None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:59.804283 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae6d60-13e6-442e-979d-608fcc2102fc None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:59.804629 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae6d60-13e6-442e-979d-608fcc2102fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:59.825639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-55ae6d60-13e6-442e-979d-608fcc2102fc None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:18:59.827155 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-55ae6d60-13e6-442e-979d-608fcc2102fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:18:59.839524 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1145/2282] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:18:59.844039 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02295d42-88f3-4b85-9c92-aec569c9faa4 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:59.857086 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02295d42-88f3-4b85-9c92-aec569c9faa4 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:59.861849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2a268-9c21-4993-a3c2-8cb78331db75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:59.862577 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2a268-9c21-4993-a3c2-8cb78331db75 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:59.862871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2a268-9c21-4993-a3c2-8cb78331db75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:59.863291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2a268-9c21-4993-a3c2-8cb78331db75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:59.864039 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2a268-9c21-4993-a3c2-8cb78331db75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:18:59.864348 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2a268-9c21-4993-a3c2-8cb78331db75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:59.865995 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2a268-9c21-4993-a3c2-8cb78331db75 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:18:59.882434 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1138/2283] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:18:59.883601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae6d60-13e6-442e-979d-608fcc2102fc None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:18:59.893201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae6d60-13e6-442e-979d-608fcc2102fc None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:18:59.898983 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3117a0f-0247-4d06-a72f-2b981832acdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:18:59.900808 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3117a0f-0247-4d06-a72f-2b981832acdd admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:18:59.900808 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3117a0f-0247-4d06-a72f-2b981832acdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:18:59.900808 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3117a0f-0247-4d06-a72f-2b981832acdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:18:59.901127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3117a0f-0247-4d06-a72f-2b981832acdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:18:59.903436 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3117a0f-0247-4d06-a72f-2b981832acdd admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:00.003287 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2a268-9c21-4993-a3c2-8cb78331db75 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:00.004727 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1146/2284] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:18:59 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:19:00.014613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2a268-9c21-4993-a3c2-8cb78331db75 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:00.088726 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5ac1c-e9c5-4db6-b925-dc0b475a1cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:00.089678 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5ac1c-e9c5-4db6-b925-dc0b475a1cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:00.090192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5ac1c-e9c5-4db6-b925-dc0b475a1cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:00.090626 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5ac1c-e9c5-4db6-b925-dc0b475a1cd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:00.091498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5ac1c-e9c5-4db6-b925-dc0b475a1cd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:00.093454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5ac1c-e9c5-4db6-b925-dc0b475a1cd3 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:00.186352 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1139/2285] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:19:00 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:19:00.190953 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3117a0f-0247-4d06-a72f-2b981832acdd admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:00.202311 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3117a0f-0247-4d06-a72f-2b981832acdd admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:00.208412 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246b99c-5e09-4d42-bbfb-0e0fe21f8874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:00.209194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246b99c-5e09-4d42-bbfb-0e0fe21f8874 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:00.209612 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246b99c-5e09-4d42-bbfb-0e0fe21f8874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:00.209782 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246b99c-5e09-4d42-bbfb-0e0fe21f8874 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:00.210575 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246b99c-5e09-4d42-bbfb-0e0fe21f8874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:00.213033 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246b99c-5e09-4d42-bbfb-0e0fe21f8874 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:00.225317 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1147/2286] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:19:00.229929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5ac1c-e9c5-4db6-b925-dc0b475a1cd3 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:00.248543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5ac1c-e9c5-4db6-b925-dc0b475a1cd3 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:00.257987 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63391b62-3ac7-4b75-a679-07b92003c0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:00.259054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63391b62-3ac7-4b75-a679-07b92003c0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:00.260486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63391b62-3ac7-4b75-a679-07b92003c0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:00.261126 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63391b62-3ac7-4b75-a679-07b92003c0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:00.262250 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63391b62-3ac7-4b75-a679-07b92003c0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:00.264247 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63391b62-3ac7-4b75-a679-07b92003c0e6 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:00.275945 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1140/2287] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:19:00.279978 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246b99c-5e09-4d42-bbfb-0e0fe21f8874 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:00.291796 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246b99c-5e09-4d42-bbfb-0e0fe21f8874 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:00.304951 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d38b5-8e93-4ff0-bb82-bf5cc50a17b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:00.305967 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d38b5-8e93-4ff0-bb82-bf5cc50a17b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:00.306850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d38b5-8e93-4ff0-bb82-bf5cc50a17b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:00.307135 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d38b5-8e93-4ff0-bb82-bf5cc50a17b0 admin admin] PATH_INFO: `/v3/projects/a427f630e1a34c81865e28d9c1bd47da/users/bd9b3bd17c7649ddbbb72c4871c5de1f/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:00.324641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d38b5-8e93-4ff0-bb82-bf5cc50a17b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a427f630e1a34c81865e28d9c1bd47da, user_id=bd9b3bd17c7649ddbbb72c4871c5de1f, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:00.327569 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d38b5-8e93-4ff0-bb82-bf5cc50a17b0 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:00.426766 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1148/2288] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:19:00 2022] PUT /identity/v3/projects/a427f630e1a34c81865e28d9c1bd47da/users/bd9b3bd17c7649ddbbb72c4871c5de1f/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:19:00.433283 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf00c08-6194-4864-a958-0476319e64c4 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:00.433578 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf00c08-6194-4864-a958-0476319e64c4 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:00.433863 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf00c08-6194-4864-a958-0476319e64c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:00.465967 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-cdf00c08-6194-4864-a958-0476319e64c4 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:00.467240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-cdf00c08-6194-4864-a958-0476319e64c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:00.599385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1141/2289] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:19:00.617186 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d38b5-8e93-4ff0-bb82-bf5cc50a17b0 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:00.619646 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-0d3d38b5-8e93-4ff0-bb82-bf5cc50a17b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:00.702721 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d38b5-8e93-4ff0-bb82-bf5cc50a17b0 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:00.711562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae45167-97a1-4053-8089-cc3cd81df255 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:00.714119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae45167-97a1-4053-8089-cc3cd81df255 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:00.714119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae45167-97a1-4053-8089-cc3cd81df255 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:00.714355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae45167-97a1-4053-8089-cc3cd81df255 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:00.728953 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae45167-97a1-4053-8089-cc3cd81df255 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:00.730469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae45167-97a1-4053-8089-cc3cd81df255 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:00.741021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1149/2290] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:19:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:19:02.842507 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c28949-86e6-4ce7-bc54-ab41870b4ff8 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:02.843037 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c28949-86e6-4ce7-bc54-ab41870b4ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:02.843215 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c28949-86e6-4ce7-bc54-ab41870b4ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:02.909692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-b3c28949-86e6-4ce7-bc54-ab41870b4ff8 None None] MFA Rules not processed for user `bd9b3bd17c7649ddbbb72c4871c5de1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:02.911036 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b3c28949-86e6-4ce7-bc54-ab41870b4ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:02.937576 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a683c-3583-499d-b2c3-878e2d3f6183 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:02.938415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a683c-3583-499d-b2c3-878e2d3f6183 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:02.938843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a683c-3583-499d-b2c3-878e2d3f6183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:02.985256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-920a683c-3583-499d-b2c3-878e2d3f6183 None None] MFA Rules not processed for user `c91839388e054c0cbe7c8e7bbb35c240`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:02.987306 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-920a683c-3583-499d-b2c3-878e2d3f6183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:03.040966 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1142/2291] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:19:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:19:03.045180 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c28949-86e6-4ce7-bc54-ab41870b4ff8 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:03.071403 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c28949-86e6-4ce7-bc54-ab41870b4ff8 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:03.076962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99964b34-e79c-43ef-bcff-6638a37df77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:03.077634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99964b34-e79c-43ef-bcff-6638a37df77e admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:03.077921 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99964b34-e79c-43ef-bcff-6638a37df77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:03.078213 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99964b34-e79c-43ef-bcff-6638a37df77e admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:03.078918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99964b34-e79c-43ef-bcff-6638a37df77e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:03.080954 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99964b34-e79c-43ef-bcff-6638a37df77e admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:03.149357 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1150/2292] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:19:02 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:19:03.164522 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a683c-3583-499d-b2c3-878e2d3f6183 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:03.167876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-920a683c-3583-499d-b2c3-878e2d3f6183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:03.223282 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1143/2293] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:19:03 2022] POST /identity/v3/projects => generated 371 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:19:03.227661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99964b34-e79c-43ef-bcff-6638a37df77e admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:03.319119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a683c-3583-499d-b2c3-878e2d3f6183 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:03.344252 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99964b34-e79c-43ef-bcff-6638a37df77e admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:03.350395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e7311-2b8e-4464-8b70-023b01e04085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:03.351086 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e7311-2b8e-4464-8b70-023b01e04085 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:03.351784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e7311-2b8e-4464-8b70-023b01e04085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:03.352070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e7311-2b8e-4464-8b70-023b01e04085 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:03.353254 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e7311-2b8e-4464-8b70-023b01e04085 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:03.359512 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e7311-2b8e-4464-8b70-023b01e04085 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:03.406674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d0003-89a6-4056-bb58-fe7f596f212a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:03.407333 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d0003-89a6-4056-bb58-fe7f596f212a service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:03.407621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d0003-89a6-4056-bb58-fe7f596f212a service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:03.407898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d0003-89a6-4056-bb58-fe7f596f212a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:03.410648 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-834d0003-89a6-4056-bb58-fe7f596f212a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:03.480727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1144/2294] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:19:03 2022] POST /identity/v3/users => generated 292 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:19:03.484718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e7311-2b8e-4464-8b70-023b01e04085 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:03.490366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d0003-89a6-4056-bb58-fe7f596f212a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:03.492038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d0003-89a6-4056-bb58-fe7f596f212a service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:03.503764 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e7311-2b8e-4464-8b70-023b01e04085 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:03.514710 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1151/2295] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:19:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:19:03.519691 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1bc58-1a63-4544-9564-1e93d7b8a257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:03.520452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1bc58-1a63-4544-9564-1e93d7b8a257 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:03.520832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1bc58-1a63-4544-9564-1e93d7b8a257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:03.521169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1bc58-1a63-4544-9564-1e93d7b8a257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:03.521987 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1bc58-1a63-4544-9564-1e93d7b8a257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:03.523736 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1bc58-1a63-4544-9564-1e93d7b8a257 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:03.538179 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1145/2296] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:19:03.539420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d0003-89a6-4056-bb58-fe7f596f212a service nova] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:03.553053 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d0003-89a6-4056-bb58-fe7f596f212a service nova] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:03.559173 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e37aa-896f-4d47-9910-28be3d10e5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:03.560649 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e37aa-896f-4d47-9910-28be3d10e5e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:03.560932 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e37aa-896f-4d47-9910-28be3d10e5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:03.561216 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e37aa-896f-4d47-9910-28be3d10e5e2 admin admin] PATH_INFO: `/v3/projects/7ecac739265442c6a5c837819acfd57a/users/833c629aa2194b2c909789c383a428b7/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:03.578326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e37aa-896f-4d47-9910-28be3d10e5e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ecac739265442c6a5c837819acfd57a, user_id=833c629aa2194b2c909789c383a428b7, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:03.581140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e37aa-896f-4d47-9910-28be3d10e5e2 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:03.681824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1152/2297] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:19:03 2022] PUT /identity/v3/projects/7ecac739265442c6a5c837819acfd57a/users/833c629aa2194b2c909789c383a428b7/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:19:03.685708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1bc58-1a63-4544-9564-1e93d7b8a257 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:03.696160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1bc58-1a63-4544-9564-1e93d7b8a257 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:03.767690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef8877-b946-40ec-a908-18c6630d4b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:03.768489 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef8877-b946-40ec-a908-18c6630d4b25 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:03.768649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef8877-b946-40ec-a908-18c6630d4b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:03.768926 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef8877-b946-40ec-a908-18c6630d4b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:03.769667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef8877-b946-40ec-a908-18c6630d4b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:03.772331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef8877-b946-40ec-a908-18c6630d4b25 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:03.792958 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1146/2298] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:19:03.797450 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e37aa-896f-4d47-9910-28be3d10e5e2 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:03.810618 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e37aa-896f-4d47-9910-28be3d10e5e2 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:03.816651 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08437fc4-efcd-45c0-9029-838b4ee3eb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:03.817549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08437fc4-efcd-45c0-9029-838b4ee3eb96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:03.817828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08437fc4-efcd-45c0-9029-838b4ee3eb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:03.818116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08437fc4-efcd-45c0-9029-838b4ee3eb96 admin admin] PATH_INFO: `/v3/projects/7ecac739265442c6a5c837819acfd57a/users/833c629aa2194b2c909789c383a428b7/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:03.820294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08437fc4-efcd-45c0-9029-838b4ee3eb96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ecac739265442c6a5c837819acfd57a, user_id=833c629aa2194b2c909789c383a428b7, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:03.824117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08437fc4-efcd-45c0-9029-838b4ee3eb96 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:03.839390 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1153/2299] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:19:03 2022] PUT /identity/v3/projects/7ecac739265442c6a5c837819acfd57a/users/833c629aa2194b2c909789c383a428b7/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:19:05.561924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab01b2a-70c6-4b91-8826-b02d20f3c1e4 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:05.562915 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab01b2a-70c6-4b91-8826-b02d20f3c1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:05.563208 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab01b2a-70c6-4b91-8826-b02d20f3c1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:05.623781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-6ab01b2a-70c6-4b91-8826-b02d20f3c1e4 None None] MFA Rules not processed for user `833c629aa2194b2c909789c383a428b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:05.625898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-6ab01b2a-70c6-4b91-8826-b02d20f3c1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:05.716006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1147/2300] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:19:05 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:19:05.729070 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08437fc4-efcd-45c0-9029-838b4ee3eb96 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:05.730978 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-08437fc4-efcd-45c0-9029-838b4ee3eb96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:05.864941 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08437fc4-efcd-45c0-9029-838b4ee3eb96 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:05.871130 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14c09a-0b5d-4721-b4ae-51ea99089591 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:05.871801 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14c09a-0b5d-4721-b4ae-51ea99089591 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:05.872122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14c09a-0b5d-4721-b4ae-51ea99089591 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:05.872406 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14c09a-0b5d-4721-b4ae-51ea99089591 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:05.950840 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14c09a-0b5d-4721-b4ae-51ea99089591 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:05.952404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14c09a-0b5d-4721-b4ae-51ea99089591 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:05.974867 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1154/2301] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:19:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:19:06.626659 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7812e63-5ef5-4b7a-a4f7-2483abee4d6c None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:06.627173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7812e63-5ef5-4b7a-a4f7-2483abee4d6c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:06.627253 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7812e63-5ef5-4b7a-a4f7-2483abee4d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:06.648474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-b7812e63-5ef5-4b7a-a4f7-2483abee4d6c None None] MFA Rules not processed for user `f213f79341ab4462b3d51352ccb9de2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:06.650170 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b7812e63-5ef5-4b7a-a4f7-2483abee4d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:06.719500 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1148/2302] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:19:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:19:07.733152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14c09a-0b5d-4721-b4ae-51ea99089591 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:07.734738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-1f14c09a-0b5d-4721-b4ae-51ea99089591 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:07.747839 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14c09a-0b5d-4721-b4ae-51ea99089591 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:07.752541 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cd59d-e8e5-4c16-855e-3a6bcd5a9341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:07.753208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cd59d-e8e5-4c16-855e-3a6bcd5a9341 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:07.753494 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cd59d-e8e5-4c16-855e-3a6bcd5a9341 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:07.753772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cd59d-e8e5-4c16-855e-3a6bcd5a9341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:07.771935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1cd59d-e8e5-4c16-855e-3a6bcd5a9341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:07.773844 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1cd59d-e8e5-4c16-855e-3a6bcd5a9341 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:07.786495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1155/2303] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:19:07 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:19:10.320754 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7812e63-5ef5-4b7a-a4f7-2483abee4d6c None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:10.382676 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7812e63-5ef5-4b7a-a4f7-2483abee4d6c None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:10.388718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ded3d-8882-4c97-8396-540a4c3d113d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:10.389954 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ded3d-8882-4c97-8396-540a4c3d113d service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:10.389954 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ded3d-8882-4c97-8396-540a4c3d113d service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:10.390495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ded3d-8882-4c97-8396-540a4c3d113d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:10.399062 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-450ded3d-8882-4c97-8396-540a4c3d113d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:10.492039 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ded3d-8882-4c97-8396-540a4c3d113d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:10.493484 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ded3d-8882-4c97-8396-540a4c3d113d service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:10.504077 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1149/2304] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:19:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:19:11.559146 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ba211-5b35-46c0-8d80-9e7dd20c9bbd None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:11.559718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ba211-5b35-46c0-8d80-9e7dd20c9bbd None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:11.559718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ba211-5b35-46c0-8d80-9e7dd20c9bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:11.638190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-4c7ba211-5b35-46c0-8d80-9e7dd20c9bbd None None] MFA Rules not processed for user `7ec2605fd1b84e0ea134b943ed33a605`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:11.639673 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4c7ba211-5b35-46c0-8d80-9e7dd20c9bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:11.729084 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1156/2305] 10.222.0.65 () {58 vars in 1009 bytes} [Tue May 10 20:19:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:19:11.759859 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ded3d-8882-4c97-8396-540a4c3d113d service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:11.761321 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-450ded3d-8882-4c97-8396-540a4c3d113d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:11.890809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ded3d-8882-4c97-8396-540a4c3d113d service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:11.904120 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6748c0-0af2-4c89-aa6a-28925e0e5170 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d10b258a53484da22950e2b1db2e41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:11.905391 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6748c0-0af2-4c89-aa6a-28925e0e5170 service swift] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:11.905941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6748c0-0af2-4c89-aa6a-28925e0e5170 service swift] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:11.906418 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6748c0-0af2-4c89-aa6a-28925e0e5170 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:11.923670 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6748c0-0af2-4c89-aa6a-28925e0e5170 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:11.927555 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6748c0-0af2-4c89-aa6a-28925e0e5170 service swift] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:11.950781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1150/2306] 10.222.0.65 () {60 vars in 1384 bytes} [Tue May 10 20:19:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 20:19:14.240961 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ba211-5b35-46c0-8d80-9e7dd20c9bbd None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:14.249367 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4c7ba211-5b35-46c0-8d80-9e7dd20c9bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:14.377886 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ba211-5b35-46c0-8d80-9e7dd20c9bbd None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:14.383456 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85271678-fef9-48a1-9928-2e6e2f37201a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:14.384471 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85271678-fef9-48a1-9928-2e6e2f37201a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:14.397206 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85271678-fef9-48a1-9928-2e6e2f37201a admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:14.397206 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85271678-fef9-48a1-9928-2e6e2f37201a admin admin] PATH_INFO: `/v3/users/b5e37bdf95e04a5f8abc2c6dda2683ea` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:14.397206 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85271678-fef9-48a1-9928-2e6e2f37201a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5e37bdf95e04a5f8abc2c6dda2683ea)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:14.397206 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85271678-fef9-48a1-9928-2e6e2f37201a admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:14.676269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-85271678-fef9-48a1-9928-2e6e2f37201a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5e37bdf95e04a5f8abc2c6dda2683ea'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:14.676269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-85271678-fef9-48a1-9928-2e6e2f37201a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5e37bdf95e04a5f8abc2c6dda2683ea'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:14.676269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-85271678-fef9-48a1-9928-2e6e2f37201a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5e37bdf95e04a5f8abc2c6dda2683ea'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:14.718527 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-85271678-fef9-48a1-9928-2e6e2f37201a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5e37bdf95e04a5f8abc2c6dda2683ea'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:14.728669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-85271678-fef9-48a1-9928-2e6e2f37201a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5e37bdf95e04a5f8abc2c6dda2683ea'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:14.887648 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1157/2307] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:19:14 2022] DELETE /identity/v3/users/b5e37bdf95e04a5f8abc2c6dda2683ea => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:19:14.892186 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139c4f5-3b18-412d-84ca-e3155841dca1 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:14.893073 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139c4f5-3b18-412d-84ca-e3155841dca1 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:14.893515 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139c4f5-3b18-412d-84ca-e3155841dca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:14.956693 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-b139c4f5-3b18-412d-84ca-e3155841dca1 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:14.960461 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b139c4f5-3b18-412d-84ca-e3155841dca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:15.109481 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1151/2308] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:19:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:19:15.157532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85271678-fef9-48a1-9928-2e6e2f37201a admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:15.162289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-85271678-fef9-48a1-9928-2e6e2f37201a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:15.310391 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85271678-fef9-48a1-9928-2e6e2f37201a admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:15.325702 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa5884-85d9-47d1-9327-aa97eabf19dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:15.332582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa5884-85d9-47d1-9327-aa97eabf19dc service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:15.338947 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa5884-85d9-47d1-9327-aa97eabf19dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:15.338947 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa5884-85d9-47d1-9327-aa97eabf19dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:15.380126 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa5884-85d9-47d1-9327-aa97eabf19dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:15.397114 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa5884-85d9-47d1-9327-aa97eabf19dc service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:15.453105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1158/2309] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:19:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:19:17.046122 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139c4f5-3b18-412d-84ca-e3155841dca1 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:17.048378 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b139c4f5-3b18-412d-84ca-e3155841dca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:17.065821 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139c4f5-3b18-412d-84ca-e3155841dca1 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:17.074486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38370776-24ac-4361-94e2-23125b8c00e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:17.075555 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38370776-24ac-4361-94e2-23125b8c00e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:17.075813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38370776-24ac-4361-94e2-23125b8c00e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:17.076093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38370776-24ac-4361-94e2-23125b8c00e8 admin admin] PATH_INFO: `/v3/projects/c2077eff313d40b195705c10603825a8` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:17.077694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38370776-24ac-4361-94e2-23125b8c00e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2077eff313d40b195705c10603825a8)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:17.079863 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38370776-24ac-4361-94e2-23125b8c00e8 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:17.276445 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-38370776-24ac-4361-94e2-23125b8c00e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2077eff313d40b195705c10603825a8', 'request_id': 'req-38370776-24ac-4361-94e2-23125b8c00e8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:17.352104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-38370776-24ac-4361-94e2-23125b8c00e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2077eff313d40b195705c10603825a8', 'request_id': 'req-38370776-24ac-4361-94e2-23125b8c00e8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:17.367294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-38370776-24ac-4361-94e2-23125b8c00e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2077eff313d40b195705c10603825a8', 'request_id': 'req-38370776-24ac-4361-94e2-23125b8c00e8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:17.380149 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-38370776-24ac-4361-94e2-23125b8c00e8 admin admin] The token cache is being invalidate because project c2077eff313d40b195705c10603825a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:19:17.380719 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-38370776-24ac-4361-94e2-23125b8c00e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:17.381587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-38370776-24ac-4361-94e2-23125b8c00e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:17.383884 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1152/2310] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:19:17 2022] DELETE /identity/v3/projects/c2077eff313d40b195705c10603825a8 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:19:17.397581 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd64b6-ce57-42cc-9075-e7c9425389e9 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:17.398533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd64b6-ce57-42cc-9075-e7c9425389e9 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:17.398962 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd64b6-ce57-42cc-9075-e7c9425389e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:17.430215 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-08bd64b6-ce57-42cc-9075-e7c9425389e9 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:17.431771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-08bd64b6-ce57-42cc-9075-e7c9425389e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:17.559807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1159/2311] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:19:17.568330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc8f0d-c1f7-4ba5-99ac-8d683a1ef70e None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:17.568625 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc8f0d-c1f7-4ba5-99ac-8d683a1ef70e None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:17.568902 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc8f0d-c1f7-4ba5-99ac-8d683a1ef70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:17.599980 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-1cbc8f0d-c1f7-4ba5-99ac-8d683a1ef70e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:17.601700 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1cbc8f0d-c1f7-4ba5-99ac-8d683a1ef70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:17.633877 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1153/2312] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:19:17.639951 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd64b6-ce57-42cc-9075-e7c9425389e9 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:17.669666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd64b6-ce57-42cc-9075-e7c9425389e9 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:17.692993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62c215-e9ef-4e30-ad00-cac5d79ced4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:17.696997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62c215-e9ef-4e30-ad00-cac5d79ced4e admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:17.697563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62c215-e9ef-4e30-ad00-cac5d79ced4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:17.699678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62c215-e9ef-4e30-ad00-cac5d79ced4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:17.702214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62c215-e9ef-4e30-ad00-cac5d79ced4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:19:17.702800 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62c215-e9ef-4e30-ad00-cac5d79ced4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:17.707504 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62c215-e9ef-4e30-ad00-cac5d79ced4e admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:17.734523 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1160/2313] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:19:17.739188 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc8f0d-c1f7-4ba5-99ac-8d683a1ef70e None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:17.758486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc8f0d-c1f7-4ba5-99ac-8d683a1ef70e None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:17.785523 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5903366-92af-4a7c-b168-8146242abee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:17.786289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5903366-92af-4a7c-b168-8146242abee1 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:17.786641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5903366-92af-4a7c-b168-8146242abee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:17.786939 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5903366-92af-4a7c-b168-8146242abee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:17.787648 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5903366-92af-4a7c-b168-8146242abee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:17.791859 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5903366-92af-4a7c-b168-8146242abee1 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:17.975626 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1154/2314] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:19:17 2022] POST /identity/v3/projects => generated 377 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:19:17.976895 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62c215-e9ef-4e30-ad00-cac5d79ced4e admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:17.991505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62c215-e9ef-4e30-ad00-cac5d79ced4e admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:18.089272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c53395-5c0a-4f51-831a-deb8fa385d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:18.089918 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c53395-5c0a-4f51-831a-deb8fa385d8e admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:18.090322 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c53395-5c0a-4f51-831a-deb8fa385d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:18.090733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c53395-5c0a-4f51-831a-deb8fa385d8e admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:18.091687 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c53395-5c0a-4f51-831a-deb8fa385d8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:18.094552 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c53395-5c0a-4f51-831a-deb8fa385d8e admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:18.164072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1161/2315] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:19:17 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:19:18.168595 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5903366-92af-4a7c-b168-8146242abee1 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:18.185859 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5903366-92af-4a7c-b168-8146242abee1 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:18.202851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614c6e8-f994-437e-a243-dc1a98e9ea12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:18.203647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614c6e8-f994-437e-a243-dc1a98e9ea12 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:18.203926 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614c6e8-f994-437e-a243-dc1a98e9ea12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:18.204205 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614c6e8-f994-437e-a243-dc1a98e9ea12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:18.205129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614c6e8-f994-437e-a243-dc1a98e9ea12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:18.206953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614c6e8-f994-437e-a243-dc1a98e9ea12 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:18.226548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1155/2316] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:19:18.227592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c53395-5c0a-4f51-831a-deb8fa385d8e admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:18.241091 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c53395-5c0a-4f51-831a-deb8fa385d8e admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:18.247072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5dedb9-e3f4-48d2-8940-45f955668ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:18.258201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5dedb9-e3f4-48d2-8940-45f955668ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:18.258201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5dedb9-e3f4-48d2-8940-45f955668ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:18.258201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5dedb9-e3f4-48d2-8940-45f955668ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:18.258201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5dedb9-e3f4-48d2-8940-45f955668ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:18.258201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5dedb9-e3f4-48d2-8940-45f955668ce9 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:18.268838 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1162/2317] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:19:18.272949 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614c6e8-f994-437e-a243-dc1a98e9ea12 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:18.285745 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614c6e8-f994-437e-a243-dc1a98e9ea12 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:18.291486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef861c-c39b-4cda-a182-12cb774c65dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:18.292658 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef861c-c39b-4cda-a182-12cb774c65dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:18.293123 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef861c-c39b-4cda-a182-12cb774c65dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:18.293554 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef861c-c39b-4cda-a182-12cb774c65dc admin admin] PATH_INFO: `/v3/projects/93dce3ce114d427895f446e0ed0f7e4f/users/46916c5b15254c529bc52ecb711fdd76/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:18.312901 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef861c-c39b-4cda-a182-12cb774c65dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=93dce3ce114d427895f446e0ed0f7e4f, user_id=46916c5b15254c529bc52ecb711fdd76, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:18.315592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef861c-c39b-4cda-a182-12cb774c65dc admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:18.452092 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1156/2318] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:19:18 2022] PUT /identity/v3/projects/93dce3ce114d427895f446e0ed0f7e4f/users/46916c5b15254c529bc52ecb711fdd76/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:19:18.459637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212e33e-d20d-4f6e-ad44-036647ad856c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:18.460501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212e33e-d20d-4f6e-ad44-036647ad856c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:18.461061 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212e33e-d20d-4f6e-ad44-036647ad856c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:18.491874 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-6212e33e-d20d-4f6e-ad44-036647ad856c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:18.493103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-6212e33e-d20d-4f6e-ad44-036647ad856c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:18.611415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1163/2319] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:19:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:19:18.628877 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef861c-c39b-4cda-a182-12cb774c65dc admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:18.630909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-e9ef861c-c39b-4cda-a182-12cb774c65dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:18.734134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef861c-c39b-4cda-a182-12cb774c65dc admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:18.744809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80db78-2eba-4838-bad6-f7e795e683b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:18.746152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80db78-2eba-4838-bad6-f7e795e683b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:18.746591 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80db78-2eba-4838-bad6-f7e795e683b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:18.747285 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80db78-2eba-4838-bad6-f7e795e683b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:18.761743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80db78-2eba-4838-bad6-f7e795e683b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:18.762972 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80db78-2eba-4838-bad6-f7e795e683b9 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:18.798332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1157/2320] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:19:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:19:24.733120 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6791643-e1bc-46a5-8f6c-b4cf3e0b5b5f None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:24.733709 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6791643-e1bc-46a5-8f6c-b4cf3e0b5b5f None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:24.733709 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6791643-e1bc-46a5-8f6c-b4cf3e0b5b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:24.755479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-d6791643-e1bc-46a5-8f6c-b4cf3e0b5b5f None None] MFA Rules not processed for user `e98e849891bf45768936056e3206d880`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:24.757069 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-d6791643-e1bc-46a5-8f6c-b4cf3e0b5b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:24.890839 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1164/2321] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:19:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:19:28.638363 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80db78-2eba-4838-bad6-f7e795e683b9 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:28.640518 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-de80db78-2eba-4838-bad6-f7e795e683b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:28.657272 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80db78-2eba-4838-bad6-f7e795e683b9 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:28.662242 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e2fc0-d5ad-49c4-b8a1-9291d3152a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:28.663128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e2fc0-d5ad-49c4-b8a1-9291d3152a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:28.663464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e2fc0-d5ad-49c4-b8a1-9291d3152a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:28.663734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e2fc0-d5ad-49c4-b8a1-9291d3152a43 admin admin] PATH_INFO: `/v3/users/bd9b3bd17c7649ddbbb72c4871c5de1f` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:28.665237 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e2fc0-d5ad-49c4-b8a1-9291d3152a43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd9b3bd17c7649ddbbb72c4871c5de1f)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:28.667218 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e2fc0-d5ad-49c4-b8a1-9291d3152a43 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:29.072471 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3f3e2fc0-d5ad-49c4-b8a1-9291d3152a43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd9b3bd17c7649ddbbb72c4871c5de1f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:29.074123 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3f3e2fc0-d5ad-49c4-b8a1-9291d3152a43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd9b3bd17c7649ddbbb72c4871c5de1f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:29.091392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3f3e2fc0-d5ad-49c4-b8a1-9291d3152a43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd9b3bd17c7649ddbbb72c4871c5de1f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:29.092224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3f3e2fc0-d5ad-49c4-b8a1-9291d3152a43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd9b3bd17c7649ddbbb72c4871c5de1f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:29.100902 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d2306-503d-496a-a7a5-c6b4f0eeba19 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:29.101501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d2306-503d-496a-a7a5-c6b4f0eeba19 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:29.101501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d2306-503d-496a-a7a5-c6b4f0eeba19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:29.117983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3f3e2fc0-d5ad-49c4-b8a1-9291d3152a43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd9b3bd17c7649ddbbb72c4871c5de1f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:29.155623 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-e06d2306-503d-496a-a7a5-c6b4f0eeba19 None None] MFA Rules not processed for user `46916c5b15254c529bc52ecb711fdd76`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:29.156854 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-e06d2306-503d-496a-a7a5-c6b4f0eeba19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:29.207765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1158/2322] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:19:28 2022] DELETE /identity/v3/users/bd9b3bd17c7649ddbbb72c4871c5de1f => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:19:29.283325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1165/2323] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:19:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:19:29.287458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e2fc0-d5ad-49c4-b8a1-9291d3152a43 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:29.289007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-3f3e2fc0-d5ad-49c4-b8a1-9291d3152a43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:29.389456 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e2fc0-d5ad-49c4-b8a1-9291d3152a43 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:29.394878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ccaca6-190a-4c3e-9641-c254c9824fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:29.396068 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccaca6-190a-4c3e-9641-c254c9824fbd admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:29.396467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccaca6-190a-4c3e-9641-c254c9824fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:29.412883 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccaca6-190a-4c3e-9641-c254c9824fbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:29.416060 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ccaca6-190a-4c3e-9641-c254c9824fbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:29.435758 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ccaca6-190a-4c3e-9641-c254c9824fbd admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:29.563242 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1159/2324] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:19:29 2022] POST /identity/v3/projects => generated 379 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:19:29.564465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d2306-503d-496a-a7a5-c6b4f0eeba19 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:29.579236 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d2306-503d-496a-a7a5-c6b4f0eeba19 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:29.650432 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ccaca6-190a-4c3e-9641-c254c9824fbd admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:29.651965 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a4ccaca6-190a-4c3e-9641-c254c9824fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:29.696321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5693d79-52cd-4dca-8894-890a9c804f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:29.697106 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5693d79-52cd-4dca-8894-890a9c804f97 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:29.697795 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5693d79-52cd-4dca-8894-890a9c804f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:29.698184 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5693d79-52cd-4dca-8894-890a9c804f97 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:29.698970 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5693d79-52cd-4dca-8894-890a9c804f97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:29.701506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5693d79-52cd-4dca-8894-890a9c804f97 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:29.729837 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ccaca6-190a-4c3e-9641-c254c9824fbd admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:29.734802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc785a6a-8554-4ee5-b4ad-9a2b426a3406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:29.735664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc785a6a-8554-4ee5-b4ad-9a2b426a3406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:29.735947 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc785a6a-8554-4ee5-b4ad-9a2b426a3406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:29.736221 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc785a6a-8554-4ee5-b4ad-9a2b426a3406 admin admin] PATH_INFO: `/v3/projects/a427f630e1a34c81865e28d9c1bd47da` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:29.737413 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc785a6a-8554-4ee5-b4ad-9a2b426a3406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a427f630e1a34c81865e28d9c1bd47da)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:29.739365 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc785a6a-8554-4ee5-b4ad-9a2b426a3406 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:29.880475 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1166/2325] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:19:29 2022] POST /identity/v3/users => generated 296 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:19:29.884706 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5693d79-52cd-4dca-8894-890a9c804f97 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:29.887559 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-dc785a6a-8554-4ee5-b4ad-9a2b426a3406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a427f630e1a34c81865e28d9c1bd47da', 'request_id': 'req-dc785a6a-8554-4ee5-b4ad-9a2b426a3406'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:29.901217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5693d79-52cd-4dca-8894-890a9c804f97 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:29.975346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3594cd-d94c-4ece-bce5-b3e4a8b63327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:29.976266 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3594cd-d94c-4ece-bce5-b3e4a8b63327 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:29.976628 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3594cd-d94c-4ece-bce5-b3e4a8b63327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:29.976912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3594cd-d94c-4ece-bce5-b3e4a8b63327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:29.977717 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3594cd-d94c-4ece-bce5-b3e4a8b63327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:29.979663 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3594cd-d94c-4ece-bce5-b3e4a8b63327 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:29.992304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1167/2326] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:19:29.995992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3594cd-d94c-4ece-bce5-b3e4a8b63327 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:30.006416 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3594cd-d94c-4ece-bce5-b3e4a8b63327 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:30.008600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-dc785a6a-8554-4ee5-b4ad-9a2b426a3406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a427f630e1a34c81865e28d9c1bd47da', 'request_id': 'req-dc785a6a-8554-4ee5-b4ad-9a2b426a3406'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:30.016614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-dc785a6a-8554-4ee5-b4ad-9a2b426a3406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a427f630e1a34c81865e28d9c1bd47da', 'request_id': 'req-dc785a6a-8554-4ee5-b4ad-9a2b426a3406'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:30.028416 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9228e4f-8b91-487a-8413-a4ed925f019f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:30.029326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9228e4f-8b91-487a-8413-a4ed925f019f admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:30.029606 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9228e4f-8b91-487a-8413-a4ed925f019f admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:30.029885 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9228e4f-8b91-487a-8413-a4ed925f019f admin admin] PATH_INFO: `/v3/projects/673c4c7cfb9d4a98a79cd5f3e3979b08/users/5283855991dd4d5390e33f57312c9531/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:30.035875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-dc785a6a-8554-4ee5-b4ad-9a2b426a3406 admin admin] The token cache is being invalidate because project a427f630e1a34c81865e28d9c1bd47da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:19:30.036173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-dc785a6a-8554-4ee5-b4ad-9a2b426a3406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:30.037624 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-dc785a6a-8554-4ee5-b4ad-9a2b426a3406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:30.040408 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1160/2327] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:19:29 2022] DELETE /identity/v3/projects/a427f630e1a34c81865e28d9c1bd47da => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:19:30.045386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc785a6a-8554-4ee5-b4ad-9a2b426a3406 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:30.048263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-dc785a6a-8554-4ee5-b4ad-9a2b426a3406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:30.058845 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9228e4f-8b91-487a-8413-a4ed925f019f admin admin] RBAC: Authorizing `identity:create_grant(project_id=673c4c7cfb9d4a98a79cd5f3e3979b08, user_id=5283855991dd4d5390e33f57312c9531, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:30.062099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9228e4f-8b91-487a-8413-a4ed925f019f admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:30.071551 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc785a6a-8554-4ee5-b4ad-9a2b426a3406 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:30.077558 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3011538-9492-4a9e-ba95-197d7de782be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:30.078747 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3011538-9492-4a9e-ba95-197d7de782be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:30.079150 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3011538-9492-4a9e-ba95-197d7de782be admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:30.079525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3011538-9492-4a9e-ba95-197d7de782be admin admin] PATH_INFO: `/v3/users/833c629aa2194b2c909789c383a428b7` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:30.081137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3011538-9492-4a9e-ba95-197d7de782be admin admin] RBAC: Authorizing `identity:delete_user(user_id=833c629aa2194b2c909789c383a428b7)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:30.083211 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3011538-9492-4a9e-ba95-197d7de782be admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:30.194190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1168/2328] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:19:29 2022] PUT /identity/v3/projects/673c4c7cfb9d4a98a79cd5f3e3979b08/users/5283855991dd4d5390e33f57312c9531/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:19:30.198222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9228e4f-8b91-487a-8413-a4ed925f019f admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:30.200507 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a9228e4f-8b91-487a-8413-a4ed925f019f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:30.269891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9228e4f-8b91-487a-8413-a4ed925f019f admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:30.275196 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd617eba-9c81-4605-9d85-6599dc36b44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:30.276103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd617eba-9c81-4605-9d85-6599dc36b44d admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:30.277287 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd617eba-9c81-4605-9d85-6599dc36b44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:30.277465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd617eba-9c81-4605-9d85-6599dc36b44d admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:30.278455 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd617eba-9c81-4605-9d85-6599dc36b44d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:30.284175 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd617eba-9c81-4605-9d85-6599dc36b44d admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:30.293970 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1169/2329] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:19:30.298528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd617eba-9c81-4605-9d85-6599dc36b44d admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:30.313723 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd617eba-9c81-4605-9d85-6599dc36b44d admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:30.322966 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae71a3e-d55c-400f-94a0-8d67def51a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:30.324167 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae71a3e-d55c-400f-94a0-8d67def51a6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:30.325214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae71a3e-d55c-400f-94a0-8d67def51a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:30.325950 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae71a3e-d55c-400f-94a0-8d67def51a6a admin admin] PATH_INFO: `/v3/projects/673c4c7cfb9d4a98a79cd5f3e3979b08/users/5283855991dd4d5390e33f57312c9531/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:30.328481 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae71a3e-d55c-400f-94a0-8d67def51a6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=673c4c7cfb9d4a98a79cd5f3e3979b08, user_id=5283855991dd4d5390e33f57312c9531, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:30.329548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-e3011538-9492-4a9e-ba95-197d7de782be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '833c629aa2194b2c909789c383a428b7'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:30.330383 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-e3011538-9492-4a9e-ba95-197d7de782be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '833c629aa2194b2c909789c383a428b7'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:30.332231 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae71a3e-d55c-400f-94a0-8d67def51a6a admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:30.361133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1170/2330] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:19:30 2022] PUT /identity/v3/projects/673c4c7cfb9d4a98a79cd5f3e3979b08/users/5283855991dd4d5390e33f57312c9531/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:19:30.372787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-e3011538-9492-4a9e-ba95-197d7de782be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '833c629aa2194b2c909789c383a428b7'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:30.373870 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-e3011538-9492-4a9e-ba95-197d7de782be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '833c629aa2194b2c909789c383a428b7'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:30.393995 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-e3011538-9492-4a9e-ba95-197d7de782be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '833c629aa2194b2c909789c383a428b7'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:30.528828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1161/2331] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:19:30 2022] DELETE /identity/v3/users/833c629aa2194b2c909789c383a428b7 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:19:31.040783 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae71a3e-d55c-400f-94a0-8d67def51a6a admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:31.044863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-eae71a3e-d55c-400f-94a0-8d67def51a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:31.196688 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae71a3e-d55c-400f-94a0-8d67def51a6a admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:31.267003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701e337-cd80-42c4-8c44-2b759c2b9876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:31.267986 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701e337-cd80-42c4-8c44-2b759c2b9876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:31.268498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701e337-cd80-42c4-8c44-2b759c2b9876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:31.268973 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701e337-cd80-42c4-8c44-2b759c2b9876 admin admin] PATH_INFO: `/v3/projects/7ecac739265442c6a5c837819acfd57a` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:31.270460 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701e337-cd80-42c4-8c44-2b759c2b9876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ecac739265442c6a5c837819acfd57a)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:31.273369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701e337-cd80-42c4-8c44-2b759c2b9876 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:31.471944 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5701e337-cd80-42c4-8c44-2b759c2b9876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ecac739265442c6a5c837819acfd57a', 'request_id': 'req-5701e337-cd80-42c4-8c44-2b759c2b9876'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:31.568313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5701e337-cd80-42c4-8c44-2b759c2b9876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ecac739265442c6a5c837819acfd57a', 'request_id': 'req-5701e337-cd80-42c4-8c44-2b759c2b9876'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:31.575596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5701e337-cd80-42c4-8c44-2b759c2b9876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ecac739265442c6a5c837819acfd57a', 'request_id': 'req-5701e337-cd80-42c4-8c44-2b759c2b9876'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:31.589567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5701e337-cd80-42c4-8c44-2b759c2b9876 admin admin] The token cache is being invalidate because project 7ecac739265442c6a5c837819acfd57a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:19:31.590163 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5701e337-cd80-42c4-8c44-2b759c2b9876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:31.591738 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5701e337-cd80-42c4-8c44-2b759c2b9876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:19:31.599684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1171/2332] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:19:31 2022] DELETE /identity/v3/projects/7ecac739265442c6a5c837819acfd57a => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:19:31.604680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2230d-d4f0-4b68-ad83-45f59d97a6c2 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:31.604680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2230d-d4f0-4b68-ad83-45f59d97a6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:31.604680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2230d-d4f0-4b68-ad83-45f59d97a6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:31.629165 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-51b2230d-d4f0-4b68-ad83-45f59d97a6c2 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:31.630554 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-51b2230d-d4f0-4b68-ad83-45f59d97a6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:31.715834 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1162/2333] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:19:31.724658 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7ba2d-4178-4c09-8edd-4d5278710a90 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:31.725027 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7ba2d-4178-4c09-8edd-4d5278710a90 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:31.725375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7ba2d-4178-4c09-8edd-4d5278710a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:31.755552 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-7fe7ba2d-4178-4c09-8edd-4d5278710a90 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:31.756263 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7fe7ba2d-4178-4c09-8edd-4d5278710a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:31.767663 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1172/2334] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:19:31.772524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2230d-d4f0-4b68-ad83-45f59d97a6c2 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:31.783279 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2230d-d4f0-4b68-ad83-45f59d97a6c2 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:31.789692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec1f41-ac38-4e20-9923-84bae3a760b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:31.792233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec1f41-ac38-4e20-9923-84bae3a760b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:31.792233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec1f41-ac38-4e20-9923-84bae3a760b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:31.792233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec1f41-ac38-4e20-9923-84bae3a760b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:31.792233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec1f41-ac38-4e20-9923-84bae3a760b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:19:31.794408 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec1f41-ac38-4e20-9923-84bae3a760b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:31.796492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec1f41-ac38-4e20-9923-84bae3a760b8 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:31.811019 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1163/2335] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:19:31.815734 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7ba2d-4178-4c09-8edd-4d5278710a90 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:31.826040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7ba2d-4178-4c09-8edd-4d5278710a90 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:31.830939 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a23fa-d7d7-4b9c-ac71-b15a1d7b16d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:31.831891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a23fa-d7d7-4b9c-ac71-b15a1d7b16d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:31.832180 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a23fa-d7d7-4b9c-ac71-b15a1d7b16d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:31.832460 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a23fa-d7d7-4b9c-ac71-b15a1d7b16d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:31.833165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a23fa-d7d7-4b9c-ac71-b15a1d7b16d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:31.835110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a23fa-d7d7-4b9c-ac71-b15a1d7b16d2 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:31.977878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1173/2336] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:19:31 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:19:31.982338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec1f41-ac38-4e20-9923-84bae3a760b8 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:32.005252 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec1f41-ac38-4e20-9923-84bae3a760b8 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:32.081572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d464d2-55ed-430e-bcb4-a0d8433a692c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:32.082323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d464d2-55ed-430e-bcb4-a0d8433a692c admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:32.082581 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d464d2-55ed-430e-bcb4-a0d8433a692c admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:32.082860 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d464d2-55ed-430e-bcb4-a0d8433a692c admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:32.083572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d464d2-55ed-430e-bcb4-a0d8433a692c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:32.086822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d464d2-55ed-430e-bcb4-a0d8433a692c admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:32.177006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1164/2337] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:19:31 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:19:32.181016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a23fa-d7d7-4b9c-ac71-b15a1d7b16d2 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:32.193596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a23fa-d7d7-4b9c-ac71-b15a1d7b16d2 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:32.199042 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9f2c7-fd6e-4bc9-ab6f-4223536cc92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:32.200460 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9f2c7-fd6e-4bc9-ab6f-4223536cc92a admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:32.200741 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9f2c7-fd6e-4bc9-ab6f-4223536cc92a admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:32.201017 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9f2c7-fd6e-4bc9-ab6f-4223536cc92a admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:32.201759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9f2c7-fd6e-4bc9-ab6f-4223536cc92a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:32.203588 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9f2c7-fd6e-4bc9-ab6f-4223536cc92a admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:32.217709 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1174/2338] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:19:32.219049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d464d2-55ed-430e-bcb4-a0d8433a692c admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:32.234920 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d464d2-55ed-430e-bcb4-a0d8433a692c admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:32.239248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113c0b8-dc74-4c80-af36-46beed3ce967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:32.240002 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113c0b8-dc74-4c80-af36-46beed3ce967 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:32.240351 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113c0b8-dc74-4c80-af36-46beed3ce967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:32.240730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113c0b8-dc74-4c80-af36-46beed3ce967 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:32.241589 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113c0b8-dc74-4c80-af36-46beed3ce967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:32.243311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113c0b8-dc74-4c80-af36-46beed3ce967 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:32.257234 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1165/2339] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:19:32.261591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9f2c7-fd6e-4bc9-ab6f-4223536cc92a admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:32.276954 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9f2c7-fd6e-4bc9-ab6f-4223536cc92a admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:32.282920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfb01a-1d66-47ab-a7e9-763c3112b8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:32.283902 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfb01a-1d66-47ab-a7e9-763c3112b8fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:32.284170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfb01a-1d66-47ab-a7e9-763c3112b8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:32.284356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfb01a-1d66-47ab-a7e9-763c3112b8fc admin admin] PATH_INFO: `/v3/projects/09ef3ef123924c56a69a88309f9029ff/users/ebcdc861aeef42d99b7b5c7b096aa32b/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:32.306956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfb01a-1d66-47ab-a7e9-763c3112b8fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=09ef3ef123924c56a69a88309f9029ff, user_id=ebcdc861aeef42d99b7b5c7b096aa32b, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:32.310484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfb01a-1d66-47ab-a7e9-763c3112b8fc admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:32.396133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1175/2340] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:19:32 2022] PUT /identity/v3/projects/09ef3ef123924c56a69a88309f9029ff/users/ebcdc861aeef42d99b7b5c7b096aa32b/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:19:32.407204 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4504f6-5a07-4b8b-a93e-c27d4dc230ee None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:32.407533 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4504f6-5a07-4b8b-a93e-c27d4dc230ee None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:32.407820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4504f6-5a07-4b8b-a93e-c27d4dc230ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:32.445585 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-2a4504f6-5a07-4b8b-a93e-c27d4dc230ee None None] MFA Rules not processed for user `ebcdc861aeef42d99b7b5c7b096aa32b`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:32.446851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-2a4504f6-5a07-4b8b-a93e-c27d4dc230ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:32.572117 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1166/2341] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:19:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:19:32.588406 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfb01a-1d66-47ab-a7e9-763c3112b8fc admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:32.591876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-5edfb01a-1d66-47ab-a7e9-763c3112b8fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:32.698051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfb01a-1d66-47ab-a7e9-763c3112b8fc admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:32.702950 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a39f7-15d3-44fc-87a6-744b13355bcc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67a252cde71a455fb1dc4bf9c47c0490', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:32.703684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a39f7-15d3-44fc-87a6-744b13355bcc service glance] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:32.704045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a39f7-15d3-44fc-87a6-744b13355bcc service glance] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:32.704394 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a39f7-15d3-44fc-87a6-744b13355bcc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:32.713516 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a39f7-15d3-44fc-87a6-744b13355bcc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:32.715060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a39f7-15d3-44fc-87a6-744b13355bcc service glance] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:32.727201 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1176/2342] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:19:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:19:37.089571 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2ba6f-a1fb-47f4-a48d-829c085f1717 None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:37.090028 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2ba6f-a1fb-47f4-a48d-829c085f1717 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:37.090149 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2ba6f-a1fb-47f4-a48d-829c085f1717 None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:37.092330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1167/2343] 10.222.0.65 () {54 vars in 921 bytes} [Tue May 10 20:19:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:19:37.098968 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bf6b1-bec6-45e0-85ae-6cc34bff97a3 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:37.103381 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bf6b1-bec6-45e0-85ae-6cc34bff97a3 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:37.103381 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bf6b1-bec6-45e0-85ae-6cc34bff97a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:37.166918 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-628bf6b1-bec6-45e0-85ae-6cc34bff97a3 None None] MFA Rules not processed for user `ae922b2da4814d10bed1fa345700ac04`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:37.168134 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-628bf6b1-bec6-45e0-85ae-6cc34bff97a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:37.260100 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1177/2344] 10.222.0.65 () {58 vars in 1018 bytes} [Tue May 10 20:19:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:19:37.279020 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2ba6f-a1fb-47f4-a48d-829c085f1717 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:37.315280 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2ba6f-a1fb-47f4-a48d-829c085f1717 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:37.326882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430217e2-32d1-4e61-b268-f4f85e9fc1a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67a252cde71a455fb1dc4bf9c47c0490', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:37.327778 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430217e2-32d1-4e61-b268-f4f85e9fc1a3 service glance] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:37.328229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430217e2-32d1-4e61-b268-f4f85e9fc1a3 service glance] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:37.329192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430217e2-32d1-4e61-b268-f4f85e9fc1a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:37.365689 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430217e2-32d1-4e61-b268-f4f85e9fc1a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:37.378484 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430217e2-32d1-4e61-b268-f4f85e9fc1a3 service glance] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:37.396652 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1168/2345] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:19:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:19:37.670099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adcdeb-f00e-4adc-b126-dce660ac5da5 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:37.670513 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adcdeb-f00e-4adc-b126-dce660ac5da5 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:37.670951 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adcdeb-f00e-4adc-b126-dce660ac5da5 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:37.672626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1178/2346] 10.222.0.65 () {54 vars in 921 bytes} [Tue May 10 20:19:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:19:37.679797 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62096229-1149-4c89-a11f-1a427042b7ad None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:37.680169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62096229-1149-4c89-a11f-1a427042b7ad None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:37.683269 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62096229-1149-4c89-a11f-1a427042b7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:37.738989 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-62096229-1149-4c89-a11f-1a427042b7ad None None] MFA Rules not processed for user `a6aabd1571d04bcfae842626fef5ed6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:37.741160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-62096229-1149-4c89-a11f-1a427042b7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:37.846451 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1169/2347] 10.222.0.65 () {58 vars in 1018 bytes} [Tue May 10 20:19:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:19:37.911562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adcdeb-f00e-4adc-b126-dce660ac5da5 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:37.914978 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-d5adcdeb-f00e-4adc-b126-dce660ac5da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:37.931253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adcdeb-f00e-4adc-b126-dce660ac5da5 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:37.936456 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b35c6-0edf-4e65-a66a-13957abade44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:37.937410 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b35c6-0edf-4e65-a66a-13957abade44 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:37.937793 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b35c6-0edf-4e65-a66a-13957abade44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:37.938216 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b35c6-0edf-4e65-a66a-13957abade44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:37.951870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b35c6-0edf-4e65-a66a-13957abade44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:37.953951 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b35c6-0edf-4e65-a66a-13957abade44 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:37.963864 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1179/2348] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:19:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:19:38.101404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f91750-2541-430c-b244-b9de5d39a60f None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:38.101860 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f91750-2541-430c-b244-b9de5d39a60f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:38.102428 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f91750-2541-430c-b244-b9de5d39a60f None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:38.103834 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1170/2349] 10.222.0.65 () {54 vars in 921 bytes} [Tue May 10 20:19:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:19:38.112958 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562d0c7-8c2a-4402-bb2e-21cdebc3d5cb None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:38.114411 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562d0c7-8c2a-4402-bb2e-21cdebc3d5cb None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:38.114930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562d0c7-8c2a-4402-bb2e-21cdebc3d5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:38.179502 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-5562d0c7-8c2a-4402-bb2e-21cdebc3d5cb None None] MFA Rules not processed for user `ae922b2da4814d10bed1fa345700ac04`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:38.180698 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-5562d0c7-8c2a-4402-bb2e-21cdebc3d5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:38.198061 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1180/2350] 10.222.0.65 () {58 vars in 1018 bytes} [Tue May 10 20:19:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:19:38.210743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f91750-2541-430c-b244-b9de5d39a60f None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:38.233618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f91750-2541-430c-b244-b9de5d39a60f None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:38.245867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e3f1a-fe85-49d6-b9e6-08a581fd81d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:38.245867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e3f1a-fe85-49d6-b9e6-08a581fd81d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:38.245867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e3f1a-fe85-49d6-b9e6-08a581fd81d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:38.246134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e3f1a-fe85-49d6-b9e6-08a581fd81d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:38.258642 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e3f1a-fe85-49d6-b9e6-08a581fd81d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:38.260163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e3f1a-fe85-49d6-b9e6-08a581fd81d8 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:38.276127 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1171/2351] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:19:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:19:39.330467 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e410872-6122-4432-989a-386982855b95 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:39.331540 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e410872-6122-4432-989a-386982855b95 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:39.331979 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e410872-6122-4432-989a-386982855b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:39.386775 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-3e410872-6122-4432-989a-386982855b95 None None] MFA Rules not processed for user `5283855991dd4d5390e33f57312c9531`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:39.396618 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3e410872-6122-4432-989a-386982855b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:39.543330 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1181/2352] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:19:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:19:39.557134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e3f1a-fe85-49d6-b9e6-08a581fd81d8 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:39.558583 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-881e3f1a-fe85-49d6-b9e6-08a581fd81d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:39.576902 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e3f1a-fe85-49d6-b9e6-08a581fd81d8 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:39.584382 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ce098-e732-4192-87d9-c025c511c966 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:39.586819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ce098-e732-4192-87d9-c025c511c966 service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:39.587937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ce098-e732-4192-87d9-c025c511c966 service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:39.588491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ce098-e732-4192-87d9-c025c511c966 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:39.594159 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f36ce098-e732-4192-87d9-c025c511c966 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:39.674317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ce098-e732-4192-87d9-c025c511c966 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:39.676055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ce098-e732-4192-87d9-c025c511c966 service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:39.697010 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1172/2353] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:19:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:19:40.884194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e410872-6122-4432-989a-386982855b95 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:40.887964 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3e410872-6122-4432-989a-386982855b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:41.036176 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e410872-6122-4432-989a-386982855b95 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:41.041890 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc48c1-cdae-4874-a1a6-4bafaaebc67d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022093dea10d46478b3016395c712d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:41.042594 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc48c1-cdae-4874-a1a6-4bafaaebc67d service placement] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:41.047901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc48c1-cdae-4874-a1a6-4bafaaebc67d service placement] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:41.047901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc48c1-cdae-4874-a1a6-4bafaaebc67d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:41.047901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-bbbc48c1-cdae-4874-a1a6-4bafaaebc67d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:41.057545 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc48c1-cdae-4874-a1a6-4bafaaebc67d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:41.059206 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc48c1-cdae-4874-a1a6-4bafaaebc67d service placement] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:41.070671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1182/2354] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:19:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:19:46.972306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98606a2a-2313-4078-a928-4c4aa9eddf45 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:46.972763 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98606a2a-2313-4078-a928-4c4aa9eddf45 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:46.972853 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98606a2a-2313-4078-a928-4c4aa9eddf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:47.020049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-98606a2a-2313-4078-a928-4c4aa9eddf45 None None] MFA Rules not processed for user `7ec2605fd1b84e0ea134b943ed33a605`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:19:47.021174 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-98606a2a-2313-4078-a928-4c4aa9eddf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:47.086729 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1173/2355] 10.222.0.65 () {58 vars in 1009 bytes} [Tue May 10 20:19:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:19:47.099225 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc48c1-cdae-4874-a1a6-4bafaaebc67d service placement] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:19:47.100652 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-bbbc48c1-cdae-4874-a1a6-4bafaaebc67d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:19:47.199429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc48c1-cdae-4874-a1a6-4bafaaebc67d service placement] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:19:47.204482 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b267d5-d78c-462e-aa4a-8a6829bb8ad8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d10b258a53484da22950e2b1db2e41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:19:47.205118 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b267d5-d78c-462e-aa4a-8a6829bb8ad8 service swift] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:19:47.205397 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b267d5-d78c-462e-aa4a-8a6829bb8ad8 service swift] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:19:47.205670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b267d5-d78c-462e-aa4a-8a6829bb8ad8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:19:47.213883 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b267d5-d78c-462e-aa4a-8a6829bb8ad8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:19:47.215477 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b267d5-d78c-462e-aa4a-8a6829bb8ad8 service swift] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:19:47.223627 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1183/2356] 10.222.0.65 () {60 vars in 1384 bytes} [Tue May 10 20:19:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 20:20:02.604605 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98606a2a-2313-4078-a928-4c4aa9eddf45 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:02.604605 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-98606a2a-2313-4078-a928-4c4aa9eddf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:02.604605 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98606a2a-2313-4078-a928-4c4aa9eddf45 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:02.606053 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc193d-70a3-4078-a945-e7366d63c72c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:02.606053 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc193d-70a3-4078-a945-e7366d63c72c service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:02.606053 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc193d-70a3-4078-a945-e7366d63c72c service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:02.606053 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc193d-70a3-4078-a945-e7366d63c72c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:02.606404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-4dbc193d-70a3-4078-a945-e7366d63c72c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:02.647524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc193d-70a3-4078-a945-e7366d63c72c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:02.649152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc193d-70a3-4078-a945-e7366d63c72c service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:02.659774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1174/2357] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:20:02 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:20:08.773047 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db480e33-c2ce-45ba-ba51-4545bc507bb9 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:08.773598 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db480e33-c2ce-45ba-ba51-4545bc507bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:08.773598 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db480e33-c2ce-45ba-ba51-4545bc507bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:08.791090 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-db480e33-c2ce-45ba-ba51-4545bc507bb9 None None] MFA Rules not processed for user `7ec2605fd1b84e0ea134b943ed33a605`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:20:08.792279 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-db480e33-c2ce-45ba-ba51-4545bc507bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:08.799321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1184/2358] 10.222.0.65 () {58 vars in 1009 bytes} [Tue May 10 20:20:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:20:08.811815 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc193d-70a3-4078-a945-e7366d63c72c service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:08.822944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc193d-70a3-4078-a945-e7366d63c72c service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:08.828376 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea78f9-5d0e-4efc-9b6f-7e4ae1e2e086 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d10b258a53484da22950e2b1db2e41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:08.829008 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea78f9-5d0e-4efc-9b6f-7e4ae1e2e086 service swift] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:08.829293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea78f9-5d0e-4efc-9b6f-7e4ae1e2e086 service swift] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:08.829570 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea78f9-5d0e-4efc-9b6f-7e4ae1e2e086 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:08.838187 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea78f9-5d0e-4efc-9b6f-7e4ae1e2e086 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:08.840937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea78f9-5d0e-4efc-9b6f-7e4ae1e2e086 service swift] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:08.848539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1175/2359] 10.222.0.65 () {60 vars in 1384 bytes} [Tue May 10 20:20:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 20:20:09.467699 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd977f35-382a-4f86-8698-aec811662d9e None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:09.468336 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd977f35-382a-4f86-8698-aec811662d9e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:09.469192 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd977f35-382a-4f86-8698-aec811662d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:09.510451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-fd977f35-382a-4f86-8698-aec811662d9e None None] MFA Rules not processed for user `46916c5b15254c529bc52ecb711fdd76`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:20:09.510451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-fd977f35-382a-4f86-8698-aec811662d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:09.519326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1185/2360] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:20:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:20:11.767779 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea78f9-5d0e-4efc-9b6f-7e4ae1e2e086 service swift] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:11.846868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea78f9-5d0e-4efc-9b6f-7e4ae1e2e086 service swift] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:11.852151 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e0ed6-168e-4895-9913-b298bba5e48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:11.853322 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e0ed6-168e-4895-9913-b298bba5e48d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:11.853730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e0ed6-168e-4895-9913-b298bba5e48d admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:11.853961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e0ed6-168e-4895-9913-b298bba5e48d admin admin] PATH_INFO: `/v3/users/ebcdc861aeef42d99b7b5c7b096aa32b` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:11.855293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e0ed6-168e-4895-9913-b298bba5e48d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebcdc861aeef42d99b7b5c7b096aa32b)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:11.857321 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e0ed6-168e-4895-9913-b298bba5e48d admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:12.042751 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-104e0ed6-168e-4895-9913-b298bba5e48d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebcdc861aeef42d99b7b5c7b096aa32b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:12.043803 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-104e0ed6-168e-4895-9913-b298bba5e48d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebcdc861aeef42d99b7b5c7b096aa32b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:12.104545 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-104e0ed6-168e-4895-9913-b298bba5e48d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebcdc861aeef42d99b7b5c7b096aa32b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:12.105553 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-104e0ed6-168e-4895-9913-b298bba5e48d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebcdc861aeef42d99b7b5c7b096aa32b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:12.136141 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-104e0ed6-168e-4895-9913-b298bba5e48d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebcdc861aeef42d99b7b5c7b096aa32b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:12.247473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1176/2361] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:20:11 2022] DELETE /identity/v3/users/ebcdc861aeef42d99b7b5c7b096aa32b => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:20:12.252274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b81daf-cbbd-4e7b-bea6-8a9a1a1cb1d6 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:12.252826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b81daf-cbbd-4e7b-bea6-8a9a1a1cb1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:12.252826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b81daf-cbbd-4e7b-bea6-8a9a1a1cb1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:12.275888 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-a5b81daf-cbbd-4e7b-bea6-8a9a1a1cb1d6 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:20:12.277046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a5b81daf-cbbd-4e7b-bea6-8a9a1a1cb1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:12.336547 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1186/2362] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:20:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:20:12.346494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e0ed6-168e-4895-9913-b298bba5e48d admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:12.347917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-104e0ed6-168e-4895-9913-b298bba5e48d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:12.413082 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e0ed6-168e-4895-9913-b298bba5e48d admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:12.416686 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832397cd-a1d9-4b42-96c3-971b456f8fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:12.417423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832397cd-a1d9-4b42-96c3-971b456f8fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:12.417725 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832397cd-a1d9-4b42-96c3-971b456f8fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:12.417991 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832397cd-a1d9-4b42-96c3-971b456f8fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:12.427159 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832397cd-a1d9-4b42-96c3-971b456f8fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:12.428635 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832397cd-a1d9-4b42-96c3-971b456f8fd2 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:12.439499 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1177/2363] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:20:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:20:13.002726 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b81daf-cbbd-4e7b-bea6-8a9a1a1cb1d6 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:13.004651 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a5b81daf-cbbd-4e7b-bea6-8a9a1a1cb1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:13.025031 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b81daf-cbbd-4e7b-bea6-8a9a1a1cb1d6 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:13.039298 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069d31b-2705-4270-b1ba-ffb5c3bd2f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:13.040462 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069d31b-2705-4270-b1ba-ffb5c3bd2f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:13.040759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069d31b-2705-4270-b1ba-ffb5c3bd2f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:13.041043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069d31b-2705-4270-b1ba-ffb5c3bd2f2c admin admin] PATH_INFO: `/v3/projects/09ef3ef123924c56a69a88309f9029ff` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:13.042367 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069d31b-2705-4270-b1ba-ffb5c3bd2f2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=09ef3ef123924c56a69a88309f9029ff)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:13.045537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069d31b-2705-4270-b1ba-ffb5c3bd2f2c admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:13.204882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-4069d31b-2705-4270-b1ba-ffb5c3bd2f2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09ef3ef123924c56a69a88309f9029ff', 'request_id': 'req-4069d31b-2705-4270-b1ba-ffb5c3bd2f2c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:13.270789 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-4069d31b-2705-4270-b1ba-ffb5c3bd2f2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09ef3ef123924c56a69a88309f9029ff', 'request_id': 'req-4069d31b-2705-4270-b1ba-ffb5c3bd2f2c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:13.284269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-4069d31b-2705-4270-b1ba-ffb5c3bd2f2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09ef3ef123924c56a69a88309f9029ff', 'request_id': 'req-4069d31b-2705-4270-b1ba-ffb5c3bd2f2c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:13.304240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-4069d31b-2705-4270-b1ba-ffb5c3bd2f2c admin admin] The token cache is being invalidate because project 09ef3ef123924c56a69a88309f9029ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:20:13.304901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-4069d31b-2705-4270-b1ba-ffb5c3bd2f2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:13.306184 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-4069d31b-2705-4270-b1ba-ffb5c3bd2f2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:13.308650 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1187/2364] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:20:13 2022] DELETE /identity/v3/projects/09ef3ef123924c56a69a88309f9029ff => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:20:13.315326 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51d0d0-5c73-4fe9-9fed-e9aaeaec9c8b None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:13.316358 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51d0d0-5c73-4fe9-9fed-e9aaeaec9c8b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:13.316819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51d0d0-5c73-4fe9-9fed-e9aaeaec9c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:13.338805 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-9a51d0d0-5c73-4fe9-9fed-e9aaeaec9c8b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:20:13.340323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-9a51d0d0-5c73-4fe9-9fed-e9aaeaec9c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:13.486008 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1178/2365] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:20:13.488288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af61b8a-dc6e-406f-a2db-ada7deb22c43 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:13.490474 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af61b8a-dc6e-406f-a2db-ada7deb22c43 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:13.490962 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af61b8a-dc6e-406f-a2db-ada7deb22c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:13.513638 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-1af61b8a-dc6e-406f-a2db-ada7deb22c43 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:20:13.515830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-1af61b8a-dc6e-406f-a2db-ada7deb22c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:13.549635 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1188/2366] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:20:13.554521 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51d0d0-5c73-4fe9-9fed-e9aaeaec9c8b None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:13.568857 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51d0d0-5c73-4fe9-9fed-e9aaeaec9c8b None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:13.588561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74959a31-c3d7-488b-9e39-44d7ef0c0825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:13.589474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74959a31-c3d7-488b-9e39-44d7ef0c0825 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:13.589961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74959a31-c3d7-488b-9e39-44d7ef0c0825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:13.590872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74959a31-c3d7-488b-9e39-44d7ef0c0825 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:13.591639 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74959a31-c3d7-488b-9e39-44d7ef0c0825 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:20:13.591953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74959a31-c3d7-488b-9e39-44d7ef0c0825 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:13.593804 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74959a31-c3d7-488b-9e39-44d7ef0c0825 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:13.607587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1179/2367] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:20:13.612671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af61b8a-dc6e-406f-a2db-ada7deb22c43 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:13.623521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af61b8a-dc6e-406f-a2db-ada7deb22c43 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:13.631396 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9a4d4-d77f-447d-b972-605eddd95558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:13.632116 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9a4d4-d77f-447d-b972-605eddd95558 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:13.632620 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9a4d4-d77f-447d-b972-605eddd95558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:13.633102 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9a4d4-d77f-447d-b972-605eddd95558 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:13.633941 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9a4d4-d77f-447d-b972-605eddd95558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:13.636427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9a4d4-d77f-447d-b972-605eddd95558 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:13.785905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1189/2368] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:20:13 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:20:13.786850 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74959a31-c3d7-488b-9e39-44d7ef0c0825 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:13.798250 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74959a31-c3d7-488b-9e39-44d7ef0c0825 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:13.862196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd6b63-b8dc-4fa1-921b-7e2390a55bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:13.863082 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd6b63-b8dc-4fa1-921b-7e2390a55bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:13.863474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd6b63-b8dc-4fa1-921b-7e2390a55bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:13.863815 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd6b63-b8dc-4fa1-921b-7e2390a55bd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:13.864621 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd6b63-b8dc-4fa1-921b-7e2390a55bd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:13.866774 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd6b63-b8dc-4fa1-921b-7e2390a55bd6 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:13.938825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1180/2369] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:20:13 2022] POST /identity/v3/users => generated 289 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:20:13.943134 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9a4d4-d77f-447d-b972-605eddd95558 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:13.956515 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9a4d4-d77f-447d-b972-605eddd95558 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:13.961792 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef53d5e-3ac0-4ce7-a2a9-036ad59c6762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:13.962775 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef53d5e-3ac0-4ce7-a2a9-036ad59c6762 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:13.963173 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef53d5e-3ac0-4ce7-a2a9-036ad59c6762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:13.963578 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef53d5e-3ac0-4ce7-a2a9-036ad59c6762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:13.964568 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef53d5e-3ac0-4ce7-a2a9-036ad59c6762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:13.966482 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef53d5e-3ac0-4ce7-a2a9-036ad59c6762 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:13.978606 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1190/2370] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:20:13.983324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd6b63-b8dc-4fa1-921b-7e2390a55bd6 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:13.994998 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd6b63-b8dc-4fa1-921b-7e2390a55bd6 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:13.999796 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04af139-53ee-419b-b427-b7fecc599e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:14.000457 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04af139-53ee-419b-b427-b7fecc599e6b admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:14.000740 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04af139-53ee-419b-b427-b7fecc599e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:14.001016 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04af139-53ee-419b-b427-b7fecc599e6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:14.001746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04af139-53ee-419b-b427-b7fecc599e6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:14.003389 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04af139-53ee-419b-b427-b7fecc599e6b admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:14.017768 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1181/2371] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:20:14.019304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef53d5e-3ac0-4ce7-a2a9-036ad59c6762 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:14.031934 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef53d5e-3ac0-4ce7-a2a9-036ad59c6762 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:14.038108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff6b16-a6ce-46f4-a42d-fe9d30ce7257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:14.039087 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff6b16-a6ce-46f4-a42d-fe9d30ce7257 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:14.039440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff6b16-a6ce-46f4-a42d-fe9d30ce7257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:14.039784 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff6b16-a6ce-46f4-a42d-fe9d30ce7257 admin admin] PATH_INFO: `/v3/projects/2947d630f28e4e68a28da46e1a8dd01c/users/9ff3cc806a284489b8fdd6982a01aae9/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:14.083647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff6b16-a6ce-46f4-a42d-fe9d30ce7257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2947d630f28e4e68a28da46e1a8dd01c, user_id=9ff3cc806a284489b8fdd6982a01aae9, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:14.088721 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff6b16-a6ce-46f4-a42d-fe9d30ce7257 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:14.174004 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1191/2372] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:20:14 2022] PUT /identity/v3/projects/2947d630f28e4e68a28da46e1a8dd01c/users/9ff3cc806a284489b8fdd6982a01aae9/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:20:14.184748 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db6ec6-0c59-4585-9705-116d3956ea6b None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:14.185400 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db6ec6-0c59-4585-9705-116d3956ea6b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:14.185927 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db6ec6-0c59-4585-9705-116d3956ea6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:14.234504 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-57db6ec6-0c59-4585-9705-116d3956ea6b None None] MFA Rules not processed for user `9ff3cc806a284489b8fdd6982a01aae9`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:20:14.235836 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-57db6ec6-0c59-4585-9705-116d3956ea6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:14.320433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1182/2373] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:20:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:20:14.333524 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff6b16-a6ce-46f4-a42d-fe9d30ce7257 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:14.336146 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-fcff6b16-a6ce-46f4-a42d-fe9d30ce7257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:14.407839 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff6b16-a6ce-46f4-a42d-fe9d30ce7257 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:14.412805 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bcc785-2364-4dd9-b355-de26dec539a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:14.413606 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcc785-2364-4dd9-b355-de26dec539a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:14.414017 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcc785-2364-4dd9-b355-de26dec539a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:14.414411 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcc785-2364-4dd9-b355-de26dec539a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:14.430954 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bcc785-2364-4dd9-b355-de26dec539a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:14.432627 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bcc785-2364-4dd9-b355-de26dec539a2 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:14.450357 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1192/2374] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:20:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:20:16.523244 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758526be-516b-4689-b796-2f40aa457d89 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:16.524129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758526be-516b-4689-b796-2f40aa457d89 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:16.524416 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758526be-516b-4689-b796-2f40aa457d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:16.562526 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-758526be-516b-4689-b796-2f40aa457d89 None None] MFA Rules not processed for user `7ec2605fd1b84e0ea134b943ed33a605`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:20:16.564027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-758526be-516b-4689-b796-2f40aa457d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:16.634942 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1183/2375] 10.222.0.65 () {58 vars in 967 bytes} [Tue May 10 20:20:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:20:16.651783 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bcc785-2364-4dd9-b355-de26dec539a2 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:16.655235 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-04bcc785-2364-4dd9-b355-de26dec539a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:16.751818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bcc785-2364-4dd9-b355-de26dec539a2 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:16.761274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0015cc6-af78-4d41-9c3e-e05961f5457e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d10b258a53484da22950e2b1db2e41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:16.767238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0015cc6-af78-4d41-9c3e-e05961f5457e service swift] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:16.767238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0015cc6-af78-4d41-9c3e-e05961f5457e service swift] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:16.767238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0015cc6-af78-4d41-9c3e-e05961f5457e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:16.784205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0015cc6-af78-4d41-9c3e-e05961f5457e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:16.786240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0015cc6-af78-4d41-9c3e-e05961f5457e service swift] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:16.799486 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1193/2376] 10.222.0.65 () {60 vars in 1384 bytes} [Tue May 10 20:20:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 20:20:27.802308 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899563a2-4daf-45d9-a0e9-2bb21f6da76a None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:27.802813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899563a2-4daf-45d9-a0e9-2bb21f6da76a None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:27.802918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899563a2-4daf-45d9-a0e9-2bb21f6da76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:27.822871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-899563a2-4daf-45d9-a0e9-2bb21f6da76a None None] MFA Rules not processed for user `7ec2605fd1b84e0ea134b943ed33a605`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:20:27.824138 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-899563a2-4daf-45d9-a0e9-2bb21f6da76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:27.831887 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1184/2377] 10.222.0.65 () {58 vars in 967 bytes} [Tue May 10 20:20:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:20:27.846772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0015cc6-af78-4d41-9c3e-e05961f5457e service swift] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:27.857756 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0015cc6-af78-4d41-9c3e-e05961f5457e service swift] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:27.863757 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf78c2-65b7-4822-98fa-53f8b68bfbcc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d10b258a53484da22950e2b1db2e41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:27.865660 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf78c2-65b7-4822-98fa-53f8b68bfbcc service swift] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:27.865660 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf78c2-65b7-4822-98fa-53f8b68bfbcc service swift] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:27.865660 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf78c2-65b7-4822-98fa-53f8b68bfbcc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:27.874549 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf78c2-65b7-4822-98fa-53f8b68bfbcc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:27.876323 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf78c2-65b7-4822-98fa-53f8b68bfbcc service swift] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:27.885310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1194/2378] 10.222.0.65 () {60 vars in 1384 bytes} [Tue May 10 20:20:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 20:20:30.900361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899563a2-4daf-45d9-a0e9-2bb21f6da76a None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:30.905794 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-899563a2-4daf-45d9-a0e9-2bb21f6da76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:30.966009 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899563a2-4daf-45d9-a0e9-2bb21f6da76a None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:30.971144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1329d179-4b95-405e-b2d0-abffb70ac1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:30.972012 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329d179-4b95-405e-b2d0-abffb70ac1af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:30.972293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329d179-4b95-405e-b2d0-abffb70ac1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:30.972596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329d179-4b95-405e-b2d0-abffb70ac1af admin admin] PATH_INFO: `/v3/users/e98e849891bf45768936056e3206d880` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:30.973797 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1329d179-4b95-405e-b2d0-abffb70ac1af admin admin] RBAC: Authorizing `identity:delete_user(user_id=e98e849891bf45768936056e3206d880)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:30.975794 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1329d179-4b95-405e-b2d0-abffb70ac1af admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:31.179805 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1329d179-4b95-405e-b2d0-abffb70ac1af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e98e849891bf45768936056e3206d880'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:31.180463 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1329d179-4b95-405e-b2d0-abffb70ac1af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e98e849891bf45768936056e3206d880'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:31.191727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1329d179-4b95-405e-b2d0-abffb70ac1af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e98e849891bf45768936056e3206d880'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:31.192362 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1329d179-4b95-405e-b2d0-abffb70ac1af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e98e849891bf45768936056e3206d880'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:31.199219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1329d179-4b95-405e-b2d0-abffb70ac1af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e98e849891bf45768936056e3206d880'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:31.275958 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1185/2379] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:20:30 2022] DELETE /identity/v3/users/e98e849891bf45768936056e3206d880 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:20:31.620866 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf78c2-65b7-4822-98fa-53f8b68bfbcc service swift] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:31.631797 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf78c2-65b7-4822-98fa-53f8b68bfbcc service swift] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:31.687436 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88bf8e-d544-4b13-8c2a-92f18e6c8160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:31.688292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88bf8e-d544-4b13-8c2a-92f18e6c8160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:31.688578 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88bf8e-d544-4b13-8c2a-92f18e6c8160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:31.689389 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88bf8e-d544-4b13-8c2a-92f18e6c8160 admin admin] PATH_INFO: `/v3/projects/752cb97b116941b7bc7ef28b2e796b1f` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:31.690605 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88bf8e-d544-4b13-8c2a-92f18e6c8160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=752cb97b116941b7bc7ef28b2e796b1f)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:31.692511 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88bf8e-d544-4b13-8c2a-92f18e6c8160 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:31.802291 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7b88bf8e-d544-4b13-8c2a-92f18e6c8160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '752cb97b116941b7bc7ef28b2e796b1f', 'request_id': 'req-7b88bf8e-d544-4b13-8c2a-92f18e6c8160'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:31.854100 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7b88bf8e-d544-4b13-8c2a-92f18e6c8160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '752cb97b116941b7bc7ef28b2e796b1f', 'request_id': 'req-7b88bf8e-d544-4b13-8c2a-92f18e6c8160'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:31.860194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7b88bf8e-d544-4b13-8c2a-92f18e6c8160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '752cb97b116941b7bc7ef28b2e796b1f', 'request_id': 'req-7b88bf8e-d544-4b13-8c2a-92f18e6c8160'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:31.872286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7b88bf8e-d544-4b13-8c2a-92f18e6c8160 admin admin] The token cache is being invalidate because project 752cb97b116941b7bc7ef28b2e796b1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:20:31.872578 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7b88bf8e-d544-4b13-8c2a-92f18e6c8160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:31.873261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7b88bf8e-d544-4b13-8c2a-92f18e6c8160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:31.874913 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1195/2380] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:20:31 2022] DELETE /identity/v3/projects/752cb97b116941b7bc7ef28b2e796b1f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:20:31.880294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9be1f-03b4-4552-ba91-a77093108d9e None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:31.880589 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9be1f-03b4-4552-ba91-a77093108d9e None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:31.880894 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9be1f-03b4-4552-ba91-a77093108d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:31.900305 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-26c9be1f-03b4-4552-ba91-a77093108d9e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:20:31.902006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-26c9be1f-03b4-4552-ba91-a77093108d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:31.963668 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1186/2381] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:20:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:20:31.971237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94322f-0600-4130-94fe-a4ebbda8551f None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:31.971528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94322f-0600-4130-94fe-a4ebbda8551f None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:31.971809 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94322f-0600-4130-94fe-a4ebbda8551f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:31.990124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-fa94322f-0600-4130-94fe-a4ebbda8551f None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:20:31.991337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-fa94322f-0600-4130-94fe-a4ebbda8551f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:31.999145 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1196/2382] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:20:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:20:32.003826 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c9be1f-03b4-4552-ba91-a77093108d9e None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:32.013362 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c9be1f-03b4-4552-ba91-a77093108d9e None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:32.017940 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169b07d-a14e-43e4-9a86-8d956d82a8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:32.018624 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169b07d-a14e-43e4-9a86-8d956d82a8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:32.018910 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169b07d-a14e-43e4-9a86-8d956d82a8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:32.019188 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169b07d-a14e-43e4-9a86-8d956d82a8e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:32.019916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169b07d-a14e-43e4-9a86-8d956d82a8e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:20:32.020225 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169b07d-a14e-43e4-9a86-8d956d82a8e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:32.021821 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169b07d-a14e-43e4-9a86-8d956d82a8e1 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:32.033029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1187/2383] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:20:32.037292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94322f-0600-4130-94fe-a4ebbda8551f None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:32.046190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94322f-0600-4130-94fe-a4ebbda8551f None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:32.051585 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e2b7f-8f2d-4754-ad0b-9ec92ed4217d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:32.052223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e2b7f-8f2d-4754-ad0b-9ec92ed4217d admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:32.052510 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e2b7f-8f2d-4754-ad0b-9ec92ed4217d admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:32.052788 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e2b7f-8f2d-4754-ad0b-9ec92ed4217d admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:32.053496 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e2b7f-8f2d-4754-ad0b-9ec92ed4217d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:32.055450 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e2b7f-8f2d-4754-ad0b-9ec92ed4217d admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:32.174454 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1197/2384] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:20:32 2022] POST /identity/v3/projects => generated 371 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:20:32.178653 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169b07d-a14e-43e4-9a86-8d956d82a8e1 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:32.189649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169b07d-a14e-43e4-9a86-8d956d82a8e1 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:32.245387 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd92dd-8bc3-4357-b245-1ed37109130d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:32.245971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd92dd-8bc3-4357-b245-1ed37109130d admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:32.246289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd92dd-8bc3-4357-b245-1ed37109130d admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:32.246568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd92dd-8bc3-4357-b245-1ed37109130d admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:32.247272 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd92dd-8bc3-4357-b245-1ed37109130d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:32.249176 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd92dd-8bc3-4357-b245-1ed37109130d admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:32.316283 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1188/2385] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:20:32 2022] POST /identity/v3/users => generated 286 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:20:32.320541 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e2b7f-8f2d-4754-ad0b-9ec92ed4217d admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:32.331026 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e2b7f-8f2d-4754-ad0b-9ec92ed4217d admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:32.335833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8938b4-980f-466e-bd22-d839250ef905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:32.336483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8938b4-980f-466e-bd22-d839250ef905 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:32.336768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8938b4-980f-466e-bd22-d839250ef905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:32.337045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8938b4-980f-466e-bd22-d839250ef905 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:32.337778 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8938b4-980f-466e-bd22-d839250ef905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:32.339714 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8938b4-980f-466e-bd22-d839250ef905 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:32.350406 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1198/2386] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:20:32.354200 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd92dd-8bc3-4357-b245-1ed37109130d admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:32.364675 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd92dd-8bc3-4357-b245-1ed37109130d admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:32.369457 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f33b6-e959-4e6f-8fd9-ac3b0b5e6199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:32.370128 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f33b6-e959-4e6f-8fd9-ac3b0b5e6199 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:32.370417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f33b6-e959-4e6f-8fd9-ac3b0b5e6199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:32.370692 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f33b6-e959-4e6f-8fd9-ac3b0b5e6199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:32.371444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f33b6-e959-4e6f-8fd9-ac3b0b5e6199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:32.373132 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f33b6-e959-4e6f-8fd9-ac3b0b5e6199 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:32.382831 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1189/2387] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:20:32.386886 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8938b4-980f-466e-bd22-d839250ef905 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:32.396574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8938b4-980f-466e-bd22-d839250ef905 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:32.401132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501d567-c95f-428a-8030-d4d85fcc571a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:32.401989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501d567-c95f-428a-8030-d4d85fcc571a admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:32.402306 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501d567-c95f-428a-8030-d4d85fcc571a admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:32.402583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501d567-c95f-428a-8030-d4d85fcc571a admin admin] PATH_INFO: `/v3/projects/2aaf1d2fe302423eb2150b28f21cafcb/users/1c7abe56ca444f7495ae57286cca544d/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:32.416997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501d567-c95f-428a-8030-d4d85fcc571a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aaf1d2fe302423eb2150b28f21cafcb, user_id=1c7abe56ca444f7495ae57286cca544d, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:32.419675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501d567-c95f-428a-8030-d4d85fcc571a admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:32.481639 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1199/2388] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:20:32 2022] PUT /identity/v3/projects/2aaf1d2fe302423eb2150b28f21cafcb/users/1c7abe56ca444f7495ae57286cca544d/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:20:32.496909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679639f5-1846-4279-b0a3-7db94692eb28 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:32.497203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679639f5-1846-4279-b0a3-7db94692eb28 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:32.497480 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679639f5-1846-4279-b0a3-7db94692eb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:32.524326 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-679639f5-1846-4279-b0a3-7db94692eb28 None None] MFA Rules not processed for user `1c7abe56ca444f7495ae57286cca544d`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:20:32.525772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-679639f5-1846-4279-b0a3-7db94692eb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:32.596907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1190/2389] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:20:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:20:32.608883 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501d567-c95f-428a-8030-d4d85fcc571a admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:32.611503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-e501d567-c95f-428a-8030-d4d85fcc571a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:32.670736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501d567-c95f-428a-8030-d4d85fcc571a admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:32.674642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d44be6-4d30-40b8-bdcc-98c206e18e73 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67a252cde71a455fb1dc4bf9c47c0490', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:32.675395 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d44be6-4d30-40b8-bdcc-98c206e18e73 service glance] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:32.675786 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d44be6-4d30-40b8-bdcc-98c206e18e73 service glance] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:32.676163 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d44be6-4d30-40b8-bdcc-98c206e18e73 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:32.685146 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d44be6-4d30-40b8-bdcc-98c206e18e73 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:32.686638 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d44be6-4d30-40b8-bdcc-98c206e18e73 service glance] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:32.695947 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1200/2390] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:20:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:20:39.130508 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679639f5-1846-4279-b0a3-7db94692eb28 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:39.131981 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-679639f5-1846-4279-b0a3-7db94692eb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:39.198906 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679639f5-1846-4279-b0a3-7db94692eb28 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:39.203442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468bccb-cc2e-4ad8-84d1-dc0f53572916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:39.204317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468bccb-cc2e-4ad8-84d1-dc0f53572916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:39.204603 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468bccb-cc2e-4ad8-84d1-dc0f53572916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:39.204936 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468bccb-cc2e-4ad8-84d1-dc0f53572916 admin admin] PATH_INFO: `/v3/users/9ff3cc806a284489b8fdd6982a01aae9` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:39.206165 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468bccb-cc2e-4ad8-84d1-dc0f53572916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ff3cc806a284489b8fdd6982a01aae9)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:39.208084 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468bccb-cc2e-4ad8-84d1-dc0f53572916 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:39.353958 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-8468bccb-cc2e-4ad8-84d1-dc0f53572916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ff3cc806a284489b8fdd6982a01aae9'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:39.354568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-8468bccb-cc2e-4ad8-84d1-dc0f53572916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ff3cc806a284489b8fdd6982a01aae9'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:39.366700 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-8468bccb-cc2e-4ad8-84d1-dc0f53572916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ff3cc806a284489b8fdd6982a01aae9'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:39.367340 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-8468bccb-cc2e-4ad8-84d1-dc0f53572916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ff3cc806a284489b8fdd6982a01aae9'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:39.373963 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-8468bccb-cc2e-4ad8-84d1-dc0f53572916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ff3cc806a284489b8fdd6982a01aae9'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:39.432146 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1191/2391] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:20:39 2022] DELETE /identity/v3/users/9ff3cc806a284489b8fdd6982a01aae9 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:20:39.436641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d6743-ff76-475f-acda-d324d11112bc None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:39.437122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d6743-ff76-475f-acda-d324d11112bc None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:39.437259 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d6743-ff76-475f-acda-d324d11112bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:39.454652 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-158d6743-ff76-475f-acda-d324d11112bc None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:20:39.456358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-158d6743-ff76-475f-acda-d324d11112bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:39.515438 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1201/2392] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:20:39.522726 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468bccb-cc2e-4ad8-84d1-dc0f53572916 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:39.524269 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8468bccb-cc2e-4ad8-84d1-dc0f53572916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:39.580858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468bccb-cc2e-4ad8-84d1-dc0f53572916 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:39.586497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec4b93-d350-4a5b-b4bd-83903a3cb9df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:39.586986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec4b93-d350-4a5b-b4bd-83903a3cb9df service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:39.587330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec4b93-d350-4a5b-b4bd-83903a3cb9df service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:39.587707 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec4b93-d350-4a5b-b4bd-83903a3cb9df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:39.604534 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec4b93-d350-4a5b-b4bd-83903a3cb9df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:39.605897 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec4b93-d350-4a5b-b4bd-83903a3cb9df service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:39.616025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1192/2393] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:20:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:20:40.138028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d6743-ff76-475f-acda-d324d11112bc None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:40.141716 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-158d6743-ff76-475f-acda-d324d11112bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:40.155551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d6743-ff76-475f-acda-d324d11112bc None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:40.161754 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227daafd-81e7-4319-8ab6-6de821841f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:40.163074 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227daafd-81e7-4319-8ab6-6de821841f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:40.163484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227daafd-81e7-4319-8ab6-6de821841f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:40.163834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227daafd-81e7-4319-8ab6-6de821841f84 admin admin] PATH_INFO: `/v3/projects/2947d630f28e4e68a28da46e1a8dd01c` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:40.165283 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227daafd-81e7-4319-8ab6-6de821841f84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2947d630f28e4e68a28da46e1a8dd01c)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:40.167263 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227daafd-81e7-4319-8ab6-6de821841f84 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:40.319088 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-227daafd-81e7-4319-8ab6-6de821841f84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2947d630f28e4e68a28da46e1a8dd01c', 'request_id': 'req-227daafd-81e7-4319-8ab6-6de821841f84'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:40.374203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-227daafd-81e7-4319-8ab6-6de821841f84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2947d630f28e4e68a28da46e1a8dd01c', 'request_id': 'req-227daafd-81e7-4319-8ab6-6de821841f84'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:40.381914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-227daafd-81e7-4319-8ab6-6de821841f84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2947d630f28e4e68a28da46e1a8dd01c', 'request_id': 'req-227daafd-81e7-4319-8ab6-6de821841f84'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:40.394022 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-227daafd-81e7-4319-8ab6-6de821841f84 admin admin] The token cache is being invalidate because project 2947d630f28e4e68a28da46e1a8dd01c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:20:40.394287 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-227daafd-81e7-4319-8ab6-6de821841f84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:40.394942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-227daafd-81e7-4319-8ab6-6de821841f84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:40.396577 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1202/2394] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:20:40 2022] DELETE /identity/v3/projects/2947d630f28e4e68a28da46e1a8dd01c => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:20:40.401576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475307a-39b1-40ed-bfff-c841af819539 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:40.402114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475307a-39b1-40ed-bfff-c841af819539 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:40.402226 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475307a-39b1-40ed-bfff-c841af819539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:40.421370 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-8475307a-39b1-40ed-bfff-c841af819539 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:20:40.422644 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8475307a-39b1-40ed-bfff-c841af819539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:40.480809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1193/2395] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:20:40.489092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2e993-9622-4e2b-a182-cae9c5874afe None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:40.489591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2e993-9622-4e2b-a182-cae9c5874afe None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:40.489697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2e993-9622-4e2b-a182-cae9c5874afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:40.508698 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-d7f2e993-9622-4e2b-a182-cae9c5874afe None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:20:40.509937 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-d7f2e993-9622-4e2b-a182-cae9c5874afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:40.518799 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1203/2396] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:20:40.523009 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475307a-39b1-40ed-bfff-c841af819539 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:40.532546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475307a-39b1-40ed-bfff-c841af819539 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:40.537547 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d016b2-11a0-49fc-b70e-e23daf9ea9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:40.538225 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d016b2-11a0-49fc-b70e-e23daf9ea9df admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:40.538512 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d016b2-11a0-49fc-b70e-e23daf9ea9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:40.538793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d016b2-11a0-49fc-b70e-e23daf9ea9df admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:40.539516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d016b2-11a0-49fc-b70e-e23daf9ea9df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:20:40.539825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d016b2-11a0-49fc-b70e-e23daf9ea9df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:40.541423 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d016b2-11a0-49fc-b70e-e23daf9ea9df admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:40.553193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1194/2397] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:20:40.557328 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2e993-9622-4e2b-a182-cae9c5874afe None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:40.566544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2e993-9622-4e2b-a182-cae9c5874afe None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:40.571191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc91bd-458e-4076-aabf-abf631374159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:40.571858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc91bd-458e-4076-aabf-abf631374159 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:40.572140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc91bd-458e-4076-aabf-abf631374159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:40.572413 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc91bd-458e-4076-aabf-abf631374159 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:40.573370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc91bd-458e-4076-aabf-abf631374159 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:40.575286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc91bd-458e-4076-aabf-abf631374159 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:40.688366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1204/2398] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:20:40 2022] POST /identity/v3/projects => generated 399 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:20:40.689450 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d016b2-11a0-49fc-b70e-e23daf9ea9df admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:40.700804 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d016b2-11a0-49fc-b70e-e23daf9ea9df admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:40.757772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c67872-b70c-46fc-83a8-3dc972ee7f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:40.758480 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c67872-b70c-46fc-83a8-3dc972ee7f46 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:40.758765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c67872-b70c-46fc-83a8-3dc972ee7f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:40.759049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c67872-b70c-46fc-83a8-3dc972ee7f46 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:40.759907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c67872-b70c-46fc-83a8-3dc972ee7f46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:40.761879 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c67872-b70c-46fc-83a8-3dc972ee7f46 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:40.826821 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1195/2399] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:20:40 2022] POST /identity/v3/users => generated 300 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:20:40.833362 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc91bd-458e-4076-aabf-abf631374159 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:40.850573 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc91bd-458e-4076-aabf-abf631374159 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:40.850573 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d66bb-b34f-4de3-a836-af651b0cdf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:40.850573 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d66bb-b34f-4de3-a836-af651b0cdf60 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:40.850573 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d66bb-b34f-4de3-a836-af651b0cdf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:40.850974 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d66bb-b34f-4de3-a836-af651b0cdf60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:40.851386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d66bb-b34f-4de3-a836-af651b0cdf60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:40.853056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d66bb-b34f-4de3-a836-af651b0cdf60 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:40.865749 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1205/2400] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:20:40.869611 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c67872-b70c-46fc-83a8-3dc972ee7f46 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:40.879758 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c67872-b70c-46fc-83a8-3dc972ee7f46 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:40.884463 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda8022-f4bf-44a7-be5c-81a01a9fb93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:40.885124 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda8022-f4bf-44a7-be5c-81a01a9fb93d admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:40.885439 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda8022-f4bf-44a7-be5c-81a01a9fb93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:40.885691 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda8022-f4bf-44a7-be5c-81a01a9fb93d admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:40.886486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda8022-f4bf-44a7-be5c-81a01a9fb93d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:40.888120 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda8022-f4bf-44a7-be5c-81a01a9fb93d admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:40.901508 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1196/2401] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:20:40.905544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d66bb-b34f-4de3-a836-af651b0cdf60 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:40.915990 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d66bb-b34f-4de3-a836-af651b0cdf60 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:40.920568 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed353d55-0342-41ab-8ba8-07e767379df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:40.921432 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed353d55-0342-41ab-8ba8-07e767379df8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:40.921714 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed353d55-0342-41ab-8ba8-07e767379df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:40.921989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed353d55-0342-41ab-8ba8-07e767379df8 admin admin] PATH_INFO: `/v3/projects/af9ea79b641d4435bde9d3160c4e267d/users/eccee8bff6ad457d91516728d48d22fc/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:40.936022 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed353d55-0342-41ab-8ba8-07e767379df8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af9ea79b641d4435bde9d3160c4e267d, user_id=eccee8bff6ad457d91516728d48d22fc, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:40.938632 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed353d55-0342-41ab-8ba8-07e767379df8 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:40.991618 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1206/2402] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:20:40 2022] PUT /identity/v3/projects/af9ea79b641d4435bde9d3160c4e267d/users/eccee8bff6ad457d91516728d48d22fc/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:20:41.002894 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d9ddd-f10a-4dab-8e13-2c7cb784e1e2 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:41.003178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d9ddd-f10a-4dab-8e13-2c7cb784e1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:41.003613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d9ddd-f10a-4dab-8e13-2c7cb784e1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:41.029484 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-7e0d9ddd-f10a-4dab-8e13-2c7cb784e1e2 None None] MFA Rules not processed for user `eccee8bff6ad457d91516728d48d22fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:20:41.030708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-7e0d9ddd-f10a-4dab-8e13-2c7cb784e1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:41.094296 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1197/2403] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:20:41 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:20:41.106210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed353d55-0342-41ab-8ba8-07e767379df8 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:41.107866 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-ed353d55-0342-41ab-8ba8-07e767379df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:41.167607 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed353d55-0342-41ab-8ba8-07e767379df8 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:41.171467 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91922d88-0528-4424-90af-2b9296a42d70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:41.172144 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91922d88-0528-4424-90af-2b9296a42d70 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:41.172424 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91922d88-0528-4424-90af-2b9296a42d70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:41.172697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91922d88-0528-4424-90af-2b9296a42d70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:41.183337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91922d88-0528-4424-90af-2b9296a42d70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:41.184770 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91922d88-0528-4424-90af-2b9296a42d70 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:41.193926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1207/2404] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:20:41 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:20:42.684705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d9ddd-f10a-4dab-8e13-2c7cb784e1e2 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:42.688145 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-7e0d9ddd-f10a-4dab-8e13-2c7cb784e1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:42.744713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d9ddd-f10a-4dab-8e13-2c7cb784e1e2 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:42.748350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9314f-2d4d-4a82-8898-304f9c11ac57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:42.749046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9314f-2d4d-4a82-8898-304f9c11ac57 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:42.749332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9314f-2d4d-4a82-8898-304f9c11ac57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:42.749602 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9314f-2d4d-4a82-8898-304f9c11ac57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:42.751525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-14a9314f-2d4d-4a82-8898-304f9c11ac57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:42.812005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9314f-2d4d-4a82-8898-304f9c11ac57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:42.813431 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9314f-2d4d-4a82-8898-304f9c11ac57 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:42.823740 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1198/2405] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:20:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:20:50.162524 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91922d88-0528-4424-90af-2b9296a42d70 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:50.223186 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91922d88-0528-4424-90af-2b9296a42d70 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:50.227755 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe884b4f-634b-4264-8ab3-fc289c0bcb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:50.228608 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe884b4f-634b-4264-8ab3-fc289c0bcb22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:50.230574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe884b4f-634b-4264-8ab3-fc289c0bcb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:50.230574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe884b4f-634b-4264-8ab3-fc289c0bcb22 admin admin] PATH_INFO: `/v3/users/eccee8bff6ad457d91516728d48d22fc` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:50.230574 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe884b4f-634b-4264-8ab3-fc289c0bcb22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eccee8bff6ad457d91516728d48d22fc)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:50.232282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe884b4f-634b-4264-8ab3-fc289c0bcb22 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:50.385479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-fe884b4f-634b-4264-8ab3-fc289c0bcb22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eccee8bff6ad457d91516728d48d22fc'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:50.386160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-fe884b4f-634b-4264-8ab3-fc289c0bcb22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eccee8bff6ad457d91516728d48d22fc'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:50.386759 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-fe884b4f-634b-4264-8ab3-fc289c0bcb22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eccee8bff6ad457d91516728d48d22fc'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:50.398784 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-fe884b4f-634b-4264-8ab3-fc289c0bcb22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eccee8bff6ad457d91516728d48d22fc'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:50.405252 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-fe884b4f-634b-4264-8ab3-fc289c0bcb22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eccee8bff6ad457d91516728d48d22fc'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:50.495328 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1208/2406] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:20:50 2022] DELETE /identity/v3/users/eccee8bff6ad457d91516728d48d22fc => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:20:50.500337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cfcdf-48dd-419a-b392-8118760a6065 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:50.500337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cfcdf-48dd-419a-b392-8118760a6065 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:50.500792 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cfcdf-48dd-419a-b392-8118760a6065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:50.519538 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-0c3cfcdf-48dd-419a-b392-8118760a6065 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:20:50.520817 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0c3cfcdf-48dd-419a-b392-8118760a6065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:50.578674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1199/2407] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:20:50.588782 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe884b4f-634b-4264-8ab3-fc289c0bcb22 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:50.590221 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-fe884b4f-634b-4264-8ab3-fc289c0bcb22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:50.645642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe884b4f-634b-4264-8ab3-fc289c0bcb22 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:50.652780 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7af9a-2017-4f26-9af9-b5015c268160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:50.652780 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7af9a-2017-4f26-9af9-b5015c268160 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:50.652780 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7af9a-2017-4f26-9af9-b5015c268160 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:50.652780 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7af9a-2017-4f26-9af9-b5015c268160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:50.659328 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7af9a-2017-4f26-9af9-b5015c268160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:50.660748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7af9a-2017-4f26-9af9-b5015c268160 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:50.670250 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1209/2408] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:20:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:20:51.040096 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cfcdf-48dd-419a-b392-8118760a6065 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:20:51.041538 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0c3cfcdf-48dd-419a-b392-8118760a6065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:51.054074 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cfcdf-48dd-419a-b392-8118760a6065 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:20:51.058862 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084aa9b-11c7-4211-b36b-696d7cdd6e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:20:51.059725 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084aa9b-11c7-4211-b36b-696d7cdd6e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:51.060013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084aa9b-11c7-4211-b36b-696d7cdd6e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:51.060750 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084aa9b-11c7-4211-b36b-696d7cdd6e32 admin admin] PATH_INFO: `/v3/projects/af9ea79b641d4435bde9d3160c4e267d` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:51.061958 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084aa9b-11c7-4211-b36b-696d7cdd6e32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af9ea79b641d4435bde9d3160c4e267d)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:20:51.064175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084aa9b-11c7-4211-b36b-696d7cdd6e32 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:20:51.163922 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-2084aa9b-11c7-4211-b36b-696d7cdd6e32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af9ea79b641d4435bde9d3160c4e267d', 'request_id': 'req-2084aa9b-11c7-4211-b36b-696d7cdd6e32'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:51.225877 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-2084aa9b-11c7-4211-b36b-696d7cdd6e32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af9ea79b641d4435bde9d3160c4e267d', 'request_id': 'req-2084aa9b-11c7-4211-b36b-696d7cdd6e32'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:51.231477 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-2084aa9b-11c7-4211-b36b-696d7cdd6e32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af9ea79b641d4435bde9d3160c4e267d', 'request_id': 'req-2084aa9b-11c7-4211-b36b-696d7cdd6e32'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:51.243032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-2084aa9b-11c7-4211-b36b-696d7cdd6e32 admin admin] The token cache is being invalidate because project af9ea79b641d4435bde9d3160c4e267d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:20:51.243319 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-2084aa9b-11c7-4211-b36b-696d7cdd6e32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:51.243955 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-2084aa9b-11c7-4211-b36b-696d7cdd6e32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:20:51.245502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1200/2409] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:20:51 2022] DELETE /identity/v3/projects/af9ea79b641d4435bde9d3160c4e267d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:20:56.964155 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa9327-3917-4475-9291-f0835600d853 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:20:56.964644 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa9327-3917-4475-9291-f0835600d853 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:20:56.964741 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa9327-3917-4475-9291-f0835600d853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:20:56.981048 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-68fa9327-3917-4475-9291-f0835600d853 None None] MFA Rules not processed for user `261d01f5b22340a09708c71be64edae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:20:56.982253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-68fa9327-3917-4475-9291-f0835600d853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:20:57.035014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1210/2410] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:20:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:21:04.885143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084aa9b-11c7-4211-b36b-696d7cdd6e32 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:21:04.886887 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-2084aa9b-11c7-4211-b36b-696d7cdd6e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:04.951260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084aa9b-11c7-4211-b36b-696d7cdd6e32 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:21:04.955941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da6529-132e-44d8-92a5-e4e7e572af8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:21:04.957307 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da6529-132e-44d8-92a5-e4e7e572af8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:21:04.957583 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da6529-132e-44d8-92a5-e4e7e572af8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:21:04.957867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da6529-132e-44d8-92a5-e4e7e572af8a admin admin] PATH_INFO: `/v3/users/1c7abe56ca444f7495ae57286cca544d` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:21:04.959085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4da6529-132e-44d8-92a5-e4e7e572af8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c7abe56ca444f7495ae57286cca544d)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:21:04.960996 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4da6529-132e-44d8-92a5-e4e7e572af8a admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:21:05.096889 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-a4da6529-132e-44d8-92a5-e4e7e572af8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c7abe56ca444f7495ae57286cca544d'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:05.098208 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-a4da6529-132e-44d8-92a5-e4e7e572af8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c7abe56ca444f7495ae57286cca544d'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:05.109889 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-a4da6529-132e-44d8-92a5-e4e7e572af8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c7abe56ca444f7495ae57286cca544d'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:05.110586 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-a4da6529-132e-44d8-92a5-e4e7e572af8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c7abe56ca444f7495ae57286cca544d'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:05.117229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-a4da6529-132e-44d8-92a5-e4e7e572af8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c7abe56ca444f7495ae57286cca544d'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:05.215055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1201/2411] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:21:04 2022] DELETE /identity/v3/users/1c7abe56ca444f7495ae57286cca544d => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:21:05.219688 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e94c3-bf69-4f71-bdf6-70d2c9574755 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:21:05.220222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e94c3-bf69-4f71-bdf6-70d2c9574755 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:21:05.220222 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e94c3-bf69-4f71-bdf6-70d2c9574755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:21:05.237880 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-a96e94c3-bf69-4f71-bdf6-70d2c9574755 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:21:05.239092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a96e94c3-bf69-4f71-bdf6-70d2c9574755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:05.295557 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1211/2412] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:21:05.305225 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da6529-132e-44d8-92a5-e4e7e572af8a admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:21:05.306646 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a4da6529-132e-44d8-92a5-e4e7e572af8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:05.362638 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da6529-132e-44d8-92a5-e4e7e572af8a admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:21:05.366322 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf2af5-f3e8-43a0-a264-672d1957803b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:21:05.366946 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf2af5-f3e8-43a0-a264-672d1957803b service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:21:05.367281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf2af5-f3e8-43a0-a264-672d1957803b service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:21:05.367509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf2af5-f3e8-43a0-a264-672d1957803b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:21:05.376366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf2af5-f3e8-43a0-a264-672d1957803b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:21:05.377819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf2af5-f3e8-43a0-a264-672d1957803b service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:21:05.387693 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1202/2413] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:21:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:21:05.811813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e94c3-bf69-4f71-bdf6-70d2c9574755 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:21:05.813238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a96e94c3-bf69-4f71-bdf6-70d2c9574755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:05.830961 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e94c3-bf69-4f71-bdf6-70d2c9574755 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:21:05.836467 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02c7e3-e2f7-45cf-ac95-80e6fd10aaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:21:05.837396 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02c7e3-e2f7-45cf-ac95-80e6fd10aaa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:21:05.837684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02c7e3-e2f7-45cf-ac95-80e6fd10aaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:21:05.837963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02c7e3-e2f7-45cf-ac95-80e6fd10aaa7 admin admin] PATH_INFO: `/v3/projects/2aaf1d2fe302423eb2150b28f21cafcb` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:21:05.839995 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02c7e3-e2f7-45cf-ac95-80e6fd10aaa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aaf1d2fe302423eb2150b28f21cafcb)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:21:05.842134 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02c7e3-e2f7-45cf-ac95-80e6fd10aaa7 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:21:05.935357 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ef02c7e3-e2f7-45cf-ac95-80e6fd10aaa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aaf1d2fe302423eb2150b28f21cafcb', 'request_id': 'req-ef02c7e3-e2f7-45cf-ac95-80e6fd10aaa7'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:05.993627 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ef02c7e3-e2f7-45cf-ac95-80e6fd10aaa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aaf1d2fe302423eb2150b28f21cafcb', 'request_id': 'req-ef02c7e3-e2f7-45cf-ac95-80e6fd10aaa7'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:05.999434 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ef02c7e3-e2f7-45cf-ac95-80e6fd10aaa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aaf1d2fe302423eb2150b28f21cafcb', 'request_id': 'req-ef02c7e3-e2f7-45cf-ac95-80e6fd10aaa7'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:06.010482 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ef02c7e3-e2f7-45cf-ac95-80e6fd10aaa7 admin admin] The token cache is being invalidate because project 2aaf1d2fe302423eb2150b28f21cafcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:21:06.010914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ef02c7e3-e2f7-45cf-ac95-80e6fd10aaa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:06.011548 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ef02c7e3-e2f7-45cf-ac95-80e6fd10aaa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:06.013079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1212/2414] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:21:05 2022] DELETE /identity/v3/projects/2aaf1d2fe302423eb2150b28f21cafcb => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:21:28.005543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7301fb-9be5-4a38-aa75-aec92e91bf29 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:21:28.006055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7301fb-9be5-4a38-aa75-aec92e91bf29 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:21:28.006200 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7301fb-9be5-4a38-aa75-aec92e91bf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:21:28.024562 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-4e7301fb-9be5-4a38-aa75-aec92e91bf29 None None] MFA Rules not processed for user `7ec2605fd1b84e0ea134b943ed33a605`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:21:28.025760 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-4e7301fb-9be5-4a38-aa75-aec92e91bf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:28.080801 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1203/2415] 10.222.0.65 () {58 vars in 967 bytes} [Tue May 10 20:21:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:21:28.209221 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02c7e3-e2f7-45cf-ac95-80e6fd10aaa7 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:21:28.210665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-ef02c7e3-e2f7-45cf-ac95-80e6fd10aaa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:28.278871 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02c7e3-e2f7-45cf-ac95-80e6fd10aaa7 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:21:28.283826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e85ae8-1ab8-4afc-b43c-dc6c48efe939 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d10b258a53484da22950e2b1db2e41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:21:28.284477 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e85ae8-1ab8-4afc-b43c-dc6c48efe939 service swift] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:21:28.284763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e85ae8-1ab8-4afc-b43c-dc6c48efe939 service swift] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:21:28.285039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e85ae8-1ab8-4afc-b43c-dc6c48efe939 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:21:28.293255 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e85ae8-1ab8-4afc-b43c-dc6c48efe939 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:21:28.294872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e85ae8-1ab8-4afc-b43c-dc6c48efe939 service swift] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:21:28.302945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1213/2416] 10.222.0.65 () {60 vars in 1384 bytes} [Tue May 10 20:21:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 20:21:31.798450 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd431e-24ef-48a0-8ed2-7428fb73a931 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:21:31.798951 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd431e-24ef-48a0-8ed2-7428fb73a931 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:21:31.799074 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd431e-24ef-48a0-8ed2-7428fb73a931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:21:31.819031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-42cd431e-24ef-48a0-8ed2-7428fb73a931 None None] MFA Rules not processed for user `8feebf7f09cb4f7d81759f8f1336dba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:21:31.820410 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-42cd431e-24ef-48a0-8ed2-7428fb73a931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:31.882783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1204/2417] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:21:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:21:36.723822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e85ae8-1ab8-4afc-b43c-dc6c48efe939 service swift] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:21:36.725902 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-80e85ae8-1ab8-4afc-b43c-dc6c48efe939 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:36.836669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e85ae8-1ab8-4afc-b43c-dc6c48efe939 service swift] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:21:36.842404 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7e063-c841-4d1d-b01a-889ac696a3c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:21:36.843355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7e063-c841-4d1d-b01a-889ac696a3c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:21:36.843798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7e063-c841-4d1d-b01a-889ac696a3c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:21:36.844311 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7e063-c841-4d1d-b01a-889ac696a3c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:21:36.846965 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-9ec7e063-c841-4d1d-b01a-889ac696a3c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:36.973869 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7e063-c841-4d1d-b01a-889ac696a3c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:21:36.975440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7e063-c841-4d1d-b01a-889ac696a3c4 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:21:36.989281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1214/2418] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:21:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:21:39.574751 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd431e-24ef-48a0-8ed2-7428fb73a931 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:21:39.575514 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-42cd431e-24ef-48a0-8ed2-7428fb73a931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:39.645052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd431e-24ef-48a0-8ed2-7428fb73a931 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:21:39.649973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b18b88-4192-4d41-a30b-07078c9cd4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:21:39.651177 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b18b88-4192-4d41-a30b-07078c9cd4ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:21:39.651468 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b18b88-4192-4d41-a30b-07078c9cd4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:21:39.651740 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b18b88-4192-4d41-a30b-07078c9cd4ad admin admin] PATH_INFO: `/v3/users/6f5b649235d646fea59e7e61e7e6d8a9` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:21:39.653267 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b18b88-4192-4d41-a30b-07078c9cd4ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f5b649235d646fea59e7e61e7e6d8a9)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:21:39.655604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b18b88-4192-4d41-a30b-07078c9cd4ad admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:21:39.825116 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-35b18b88-4192-4d41-a30b-07078c9cd4ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f5b649235d646fea59e7e61e7e6d8a9'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:39.825776 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-35b18b88-4192-4d41-a30b-07078c9cd4ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f5b649235d646fea59e7e61e7e6d8a9'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:39.837955 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-35b18b88-4192-4d41-a30b-07078c9cd4ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f5b649235d646fea59e7e61e7e6d8a9'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:39.838731 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-35b18b88-4192-4d41-a30b-07078c9cd4ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f5b649235d646fea59e7e61e7e6d8a9'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:39.845995 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-35b18b88-4192-4d41-a30b-07078c9cd4ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f5b649235d646fea59e7e61e7e6d8a9'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:39.949957 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1205/2419] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:21:39 2022] DELETE /identity/v3/users/6f5b649235d646fea59e7e61e7e6d8a9 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:21:40.259733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7e063-c841-4d1d-b01a-889ac696a3c4 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:21:40.271152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7e063-c841-4d1d-b01a-889ac696a3c4 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:21:40.331497 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1956c2d-e107-43f8-a2a7-0f9068238216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:21:40.332350 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1956c2d-e107-43f8-a2a7-0f9068238216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:21:40.332634 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1956c2d-e107-43f8-a2a7-0f9068238216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:21:40.332911 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1956c2d-e107-43f8-a2a7-0f9068238216 admin admin] PATH_INFO: `/v3/projects/b135986dfe1f499a9655ac3faee4bbbf` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:21:40.334132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1956c2d-e107-43f8-a2a7-0f9068238216 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b135986dfe1f499a9655ac3faee4bbbf)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:21:40.336108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1956c2d-e107-43f8-a2a7-0f9068238216 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:21:40.451875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e1956c2d-e107-43f8-a2a7-0f9068238216 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b135986dfe1f499a9655ac3faee4bbbf', 'request_id': 'req-e1956c2d-e107-43f8-a2a7-0f9068238216'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:40.531907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e1956c2d-e107-43f8-a2a7-0f9068238216 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b135986dfe1f499a9655ac3faee4bbbf', 'request_id': 'req-e1956c2d-e107-43f8-a2a7-0f9068238216'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:40.538206 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e1956c2d-e107-43f8-a2a7-0f9068238216 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b135986dfe1f499a9655ac3faee4bbbf', 'request_id': 'req-e1956c2d-e107-43f8-a2a7-0f9068238216'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:40.550890 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e1956c2d-e107-43f8-a2a7-0f9068238216 admin admin] The token cache is being invalidate because project b135986dfe1f499a9655ac3faee4bbbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:21:40.550890 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e1956c2d-e107-43f8-a2a7-0f9068238216 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:40.550890 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e1956c2d-e107-43f8-a2a7-0f9068238216 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:40.552489 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1215/2420] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:21:40 2022] DELETE /identity/v3/projects/b135986dfe1f499a9655ac3faee4bbbf => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:21:40.557112 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b18b88-4192-4d41-a30b-07078c9cd4ad admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:21:40.558553 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-35b18b88-4192-4d41-a30b-07078c9cd4ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:40.571716 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b18b88-4192-4d41-a30b-07078c9cd4ad admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:21:40.634536 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a9868-a4a8-4d0d-bb69-d42500700d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:21:40.635411 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a9868-a4a8-4d0d-bb69-d42500700d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:21:40.635693 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a9868-a4a8-4d0d-bb69-d42500700d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:21:40.635966 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a9868-a4a8-4d0d-bb69-d42500700d9f admin admin] PATH_INFO: `/v3/users/72111ba468f744b2a50b100e15f6c7f8` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:21:40.637148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a9868-a4a8-4d0d-bb69-d42500700d9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=72111ba468f744b2a50b100e15f6c7f8)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:21:40.639216 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a9868-a4a8-4d0d-bb69-d42500700d9f admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:21:40.836680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-946a9868-a4a8-4d0d-bb69-d42500700d9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72111ba468f744b2a50b100e15f6c7f8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:40.837334 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-946a9868-a4a8-4d0d-bb69-d42500700d9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72111ba468f744b2a50b100e15f6c7f8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:40.848764 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-946a9868-a4a8-4d0d-bb69-d42500700d9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72111ba468f744b2a50b100e15f6c7f8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:40.849398 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-946a9868-a4a8-4d0d-bb69-d42500700d9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72111ba468f744b2a50b100e15f6c7f8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:40.856695 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-946a9868-a4a8-4d0d-bb69-d42500700d9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72111ba468f744b2a50b100e15f6c7f8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:40.958572 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1956c2d-e107-43f8-a2a7-0f9068238216 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:21:40.959950 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-e1956c2d-e107-43f8-a2a7-0f9068238216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:40.960353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1206/2421] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:21:40 2022] DELETE /identity/v3/users/72111ba468f744b2a50b100e15f6c7f8 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:21:41.020650 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1956c2d-e107-43f8-a2a7-0f9068238216 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:21:41.071517 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd952b1-7035-48c2-b5eb-8787a19de316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:21:41.072172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd952b1-7035-48c2-b5eb-8787a19de316 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:21:41.072461 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd952b1-7035-48c2-b5eb-8787a19de316 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:21:41.072724 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd952b1-7035-48c2-b5eb-8787a19de316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:21:41.074678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-5bd952b1-7035-48c2-b5eb-8787a19de316 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:41.149482 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd952b1-7035-48c2-b5eb-8787a19de316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:21:41.150991 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd952b1-7035-48c2-b5eb-8787a19de316 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:21:41.163241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1216/2422] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:21:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:21:41.223913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a9868-a4a8-4d0d-bb69-d42500700d9f admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:21:41.225350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-946a9868-a4a8-4d0d-bb69-d42500700d9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:41.298346 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a9868-a4a8-4d0d-bb69-d42500700d9f admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:21:41.303810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7991d-6f3d-4ab5-82d0-3bd4e72131f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:21:41.305214 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7991d-6f3d-4ab5-82d0-3bd4e72131f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:21:41.305923 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7991d-6f3d-4ab5-82d0-3bd4e72131f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:21:41.311604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7991d-6f3d-4ab5-82d0-3bd4e72131f8 admin admin] PATH_INFO: `/v3/projects/4b5708fb61854562988f5bc2dee5b34b` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:21:41.312039 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a7991d-6f3d-4ab5-82d0-3bd4e72131f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b5708fb61854562988f5bc2dee5b34b)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:21:41.314360 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a7991d-6f3d-4ab5-82d0-3bd4e72131f8 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:21:41.440467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-69a7991d-6f3d-4ab5-82d0-3bd4e72131f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b5708fb61854562988f5bc2dee5b34b', 'request_id': 'req-69a7991d-6f3d-4ab5-82d0-3bd4e72131f8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:41.509379 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-69a7991d-6f3d-4ab5-82d0-3bd4e72131f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b5708fb61854562988f5bc2dee5b34b', 'request_id': 'req-69a7991d-6f3d-4ab5-82d0-3bd4e72131f8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:41.515438 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-69a7991d-6f3d-4ab5-82d0-3bd4e72131f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b5708fb61854562988f5bc2dee5b34b', 'request_id': 'req-69a7991d-6f3d-4ab5-82d0-3bd4e72131f8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:41.529244 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-69a7991d-6f3d-4ab5-82d0-3bd4e72131f8 admin admin] The token cache is being invalidate because project 4b5708fb61854562988f5bc2dee5b34b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:21:41.529542 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-69a7991d-6f3d-4ab5-82d0-3bd4e72131f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:41.530498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-69a7991d-6f3d-4ab5-82d0-3bd4e72131f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:21:41.532944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1207/2423] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:21:41 2022] DELETE /identity/v3/projects/4b5708fb61854562988f5bc2dee5b34b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:21:42.131145 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ca7fb-17df-49f9-92d0-1099b8fc5958 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:21:42.131723 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ca7fb-17df-49f9-92d0-1099b8fc5958 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:21:42.131723 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ca7fb-17df-49f9-92d0-1099b8fc5958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:21:42.150252 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-744ca7fb-17df-49f9-92d0-1099b8fc5958 None None] MFA Rules not processed for user `35e69662bc7648b8b3e7467cf56c7f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:21:42.151412 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-744ca7fb-17df-49f9-92d0-1099b8fc5958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:42.204338 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1217/2424] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:21:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:21:42.876437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7991d-6f3d-4ab5-82d0-3bd4e72131f8 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:21:42.877839 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-69a7991d-6f3d-4ab5-82d0-3bd4e72131f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:42.949743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7991d-6f3d-4ab5-82d0-3bd4e72131f8 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:21:42.954969 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef11ab2-f834-43f3-bb34-59c4d32ee6aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022093dea10d46478b3016395c712d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:21:42.955611 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef11ab2-f834-43f3-bb34-59c4d32ee6aa service placement] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:21:42.955898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef11ab2-f834-43f3-bb34-59c4d32ee6aa service placement] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:21:42.956174 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef11ab2-f834-43f3-bb34-59c4d32ee6aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:21:42.957929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1ef11ab2-f834-43f3-bb34-59c4d32ee6aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:42.968978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef11ab2-f834-43f3-bb34-59c4d32ee6aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:21:42.970603 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef11ab2-f834-43f3-bb34-59c4d32ee6aa service placement] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:21:42.980875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1208/2425] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:21:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:21:47.005380 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ca7fb-17df-49f9-92d0-1099b8fc5958 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:21:47.007010 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-744ca7fb-17df-49f9-92d0-1099b8fc5958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:47.093647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ca7fb-17df-49f9-92d0-1099b8fc5958 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:21:47.100755 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff98a09-5dd6-4f24-bf88-16f137deff81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:21:47.101480 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff98a09-5dd6-4f24-bf88-16f137deff81 service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:21:47.101837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff98a09-5dd6-4f24-bf88-16f137deff81 service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:21:47.102269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff98a09-5dd6-4f24-bf88-16f137deff81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:21:47.104286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-6ff98a09-5dd6-4f24-bf88-16f137deff81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:47.174819 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff98a09-5dd6-4f24-bf88-16f137deff81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:21:47.176674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff98a09-5dd6-4f24-bf88-16f137deff81 service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:21:47.187088 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1218/2426] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:21:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:21:56.493234 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef11ab2-f834-43f3-bb34-59c4d32ee6aa service placement] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:21:56.495246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1ef11ab2-f834-43f3-bb34-59c4d32ee6aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:56.550104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef11ab2-f834-43f3-bb34-59c4d32ee6aa service placement] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:21:56.553829 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba376537-1e54-4eb2-8dfb-da6cacdc58a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:21:56.554491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba376537-1e54-4eb2-8dfb-da6cacdc58a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:21:56.554775 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba376537-1e54-4eb2-8dfb-da6cacdc58a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:21:56.559499 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba376537-1e54-4eb2-8dfb-da6cacdc58a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:21:56.559499 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-ba376537-1e54-4eb2-8dfb-da6cacdc58a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:56.566606 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba376537-1e54-4eb2-8dfb-da6cacdc58a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:21:56.568028 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba376537-1e54-4eb2-8dfb-da6cacdc58a2 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:21:56.576458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1209/2427] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:21:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:21:56.793220 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff98a09-5dd6-4f24-bf88-16f137deff81 service nova] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:21:56.804171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff98a09-5dd6-4f24-bf88-16f137deff81 service nova] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:21:56.810455 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdafc39-d9af-4adb-9ad5-21f8c4871de4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:21:56.813440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdafc39-d9af-4adb-9ad5-21f8c4871de4 service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:21:56.813440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdafc39-d9af-4adb-9ad5-21f8c4871de4 service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:21:56.813440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdafc39-d9af-4adb-9ad5-21f8c4871de4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:21:56.813440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-bcdafc39-d9af-4adb-9ad5-21f8c4871de4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:21:56.824381 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdafc39-d9af-4adb-9ad5-21f8c4871de4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:21:56.825935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdafc39-d9af-4adb-9ad5-21f8c4871de4 service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:21:56.835096 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1219/2428] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:21:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:00.805266 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba376537-1e54-4eb2-8dfb-da6cacdc58a2 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:00.806727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-ba376537-1e54-4eb2-8dfb-da6cacdc58a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:00.824890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba376537-1e54-4eb2-8dfb-da6cacdc58a2 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:00.830680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c3e07-fc1d-421c-97a4-aeec508cecdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:00.832117 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c3e07-fc1d-421c-97a4-aeec508cecdc service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:00.832473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c3e07-fc1d-421c-97a4-aeec508cecdc service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:00.832765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c3e07-fc1d-421c-97a4-aeec508cecdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:00.834633 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c83c3e07-fc1d-421c-97a4-aeec508cecdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:00.849133 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c3e07-fc1d-421c-97a4-aeec508cecdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:00.851187 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c3e07-fc1d-421c-97a4-aeec508cecdc service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:00.864997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1210/2429] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:07.272341 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdafc39-d9af-4adb-9ad5-21f8c4871de4 service nova] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:07.282280 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdafc39-d9af-4adb-9ad5-21f8c4871de4 service nova] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:07.285906 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f89354-19c3-4c53-9a6a-ca4386f7bc5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:07.286595 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f89354-19c3-4c53-9a6a-ca4386f7bc5b service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:07.286881 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f89354-19c3-4c53-9a6a-ca4386f7bc5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:07.287161 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f89354-19c3-4c53-9a6a-ca4386f7bc5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:07.288960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-36f89354-19c3-4c53-9a6a-ca4386f7bc5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:07.350992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f89354-19c3-4c53-9a6a-ca4386f7bc5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:07.352419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f89354-19c3-4c53-9a6a-ca4386f7bc5b service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:07.362455 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1220/2430] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:11.106475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c3e07-fc1d-421c-97a4-aeec508cecdc service nova] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:11.108402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c83c3e07-fc1d-421c-97a4-aeec508cecdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:11.125339 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c3e07-fc1d-421c-97a4-aeec508cecdc service nova] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:11.130143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d77b1-39e6-4e9c-811a-f5e1a74873ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:11.131029 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d77b1-39e6-4e9c-811a-f5e1a74873ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:11.131312 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d77b1-39e6-4e9c-811a-f5e1a74873ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:11.131602 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d77b1-39e6-4e9c-811a-f5e1a74873ea admin admin] PATH_INFO: `/v3/users/261d01f5b22340a09708c71be64edae8` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:11.132830 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9d77b1-39e6-4e9c-811a-f5e1a74873ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=261d01f5b22340a09708c71be64edae8)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:11.134776 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9d77b1-39e6-4e9c-811a-f5e1a74873ea admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:11.294008 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-da9d77b1-39e6-4e9c-811a-f5e1a74873ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '261d01f5b22340a09708c71be64edae8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:11.294685 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-da9d77b1-39e6-4e9c-811a-f5e1a74873ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '261d01f5b22340a09708c71be64edae8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:11.306346 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-da9d77b1-39e6-4e9c-811a-f5e1a74873ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '261d01f5b22340a09708c71be64edae8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:11.307000 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-da9d77b1-39e6-4e9c-811a-f5e1a74873ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '261d01f5b22340a09708c71be64edae8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:11.314356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-da9d77b1-39e6-4e9c-811a-f5e1a74873ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '261d01f5b22340a09708c71be64edae8'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:11.390213 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1211/2431] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:22:11 2022] DELETE /identity/v3/users/261d01f5b22340a09708c71be64edae8 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:11.638013 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f89354-19c3-4c53-9a6a-ca4386f7bc5b service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:11.648007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f89354-19c3-4c53-9a6a-ca4386f7bc5b service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:11.705169 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d16a6-ae73-4027-886c-24e3043d2e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:11.706022 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d16a6-ae73-4027-886c-24e3043d2e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:11.706337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d16a6-ae73-4027-886c-24e3043d2e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:11.706613 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d16a6-ae73-4027-886c-24e3043d2e5e admin admin] PATH_INFO: `/v3/projects/9d38de3da9e3489ebbbebd4f8aeb0d49` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:11.707795 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d16a6-ae73-4027-886c-24e3043d2e5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d38de3da9e3489ebbbebd4f8aeb0d49)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:11.709688 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d16a6-ae73-4027-886c-24e3043d2e5e admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:11.824811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c39d16a6-ae73-4027-886c-24e3043d2e5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d38de3da9e3489ebbbebd4f8aeb0d49', 'request_id': 'req-c39d16a6-ae73-4027-886c-24e3043d2e5e'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:11.904397 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c39d16a6-ae73-4027-886c-24e3043d2e5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d38de3da9e3489ebbbebd4f8aeb0d49', 'request_id': 'req-c39d16a6-ae73-4027-886c-24e3043d2e5e'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:11.909860 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c39d16a6-ae73-4027-886c-24e3043d2e5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d38de3da9e3489ebbbebd4f8aeb0d49', 'request_id': 'req-c39d16a6-ae73-4027-886c-24e3043d2e5e'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:11.921232 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c39d16a6-ae73-4027-886c-24e3043d2e5e admin admin] The token cache is being invalidate because project 9d38de3da9e3489ebbbebd4f8aeb0d49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:22:11.921511 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c39d16a6-ae73-4027-886c-24e3043d2e5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:11.922209 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c39d16a6-ae73-4027-886c-24e3043d2e5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:11.923755 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1221/2432] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:22:11 2022] DELETE /identity/v3/projects/9d38de3da9e3489ebbbebd4f8aeb0d49 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:11.928767 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7900304-1171-4539-b4ea-4f7d06e75df4 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:11.929061 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7900304-1171-4539-b4ea-4f7d06e75df4 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:11.929341 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7900304-1171-4539-b4ea-4f7d06e75df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:11.947981 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-b7900304-1171-4539-b4ea-4f7d06e75df4 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:11.949222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b7900304-1171-4539-b4ea-4f7d06e75df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:12.008049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1212/2433] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:12.015556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50955b-4c99-46b5-b12b-c0f128c04d6c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:12.015854 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50955b-4c99-46b5-b12b-c0f128c04d6c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:12.016140 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50955b-4c99-46b5-b12b-c0f128c04d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:12.033940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-bc50955b-4c99-46b5-b12b-c0f128c04d6c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:12.035169 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-bc50955b-4c99-46b5-b12b-c0f128c04d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:12.043536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1222/2434] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:12.047639 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7900304-1171-4539-b4ea-4f7d06e75df4 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:12.057076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7900304-1171-4539-b4ea-4f7d06e75df4 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:12.062604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7dce35-75b3-4ca6-bd7b-6a6dd5682e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:12.063252 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dce35-75b3-4ca6-bd7b-6a6dd5682e42 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:12.063537 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dce35-75b3-4ca6-bd7b-6a6dd5682e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:12.063905 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dce35-75b3-4ca6-bd7b-6a6dd5682e42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:12.064630 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7dce35-75b3-4ca6-bd7b-6a6dd5682e42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:22:12.064953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7dce35-75b3-4ca6-bd7b-6a6dd5682e42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:12.066559 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7dce35-75b3-4ca6-bd7b-6a6dd5682e42 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:12.078711 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1213/2435] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:22:12.082263 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50955b-4c99-46b5-b12b-c0f128c04d6c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:12.091368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50955b-4c99-46b5-b12b-c0f128c04d6c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:12.096049 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6acb1-22a1-473c-97e9-00563d437dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:12.096691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6acb1-22a1-473c-97e9-00563d437dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:12.096977 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6acb1-22a1-473c-97e9-00563d437dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:12.097253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6acb1-22a1-473c-97e9-00563d437dd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:12.097983 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6acb1-22a1-473c-97e9-00563d437dd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:12.099900 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6acb1-22a1-473c-97e9-00563d437dd4 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:12.215683 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1223/2436] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:22:12 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:22:12.219791 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7dce35-75b3-4ca6-bd7b-6a6dd5682e42 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:12.229819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7dce35-75b3-4ca6-bd7b-6a6dd5682e42 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:12.286274 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcba1c-4302-4b06-a154-c70a03f49ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:12.286976 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcba1c-4302-4b06-a154-c70a03f49ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:12.287256 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcba1c-4302-4b06-a154-c70a03f49ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:12.287531 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcba1c-4302-4b06-a154-c70a03f49ef9 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:12.288236 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcba1c-4302-4b06-a154-c70a03f49ef9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:12.290281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcba1c-4302-4b06-a154-c70a03f49ef9 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:12.378719 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1214/2437] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:22:12 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:22:12.382846 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6acb1-22a1-473c-97e9-00563d437dd4 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:12.393771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6acb1-22a1-473c-97e9-00563d437dd4 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:12.398344 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c017fb-966e-4705-a0f1-b85b6daa61f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:12.398994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c017fb-966e-4705-a0f1-b85b6daa61f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:12.399272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c017fb-966e-4705-a0f1-b85b6daa61f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:12.399542 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c017fb-966e-4705-a0f1-b85b6daa61f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:12.400335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c017fb-966e-4705-a0f1-b85b6daa61f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:12.401915 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c017fb-966e-4705-a0f1-b85b6daa61f7 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:12.412055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1224/2438] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:12.416355 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcba1c-4302-4b06-a154-c70a03f49ef9 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:12.426557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcba1c-4302-4b06-a154-c70a03f49ef9 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:12.431618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ee6bb-b9c6-4bc5-9e95-664781873019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:12.432281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ee6bb-b9c6-4bc5-9e95-664781873019 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:12.432949 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ee6bb-b9c6-4bc5-9e95-664781873019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:12.432949 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ee6bb-b9c6-4bc5-9e95-664781873019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:12.434051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ee6bb-b9c6-4bc5-9e95-664781873019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:12.436152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ee6bb-b9c6-4bc5-9e95-664781873019 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:12.451982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1215/2439] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:12.456897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c017fb-966e-4705-a0f1-b85b6daa61f7 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:12.467310 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c017fb-966e-4705-a0f1-b85b6daa61f7 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:12.472178 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5e68c-b31a-4061-8b0b-cf47e8aeb7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:12.473044 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5e68c-b31a-4061-8b0b-cf47e8aeb7d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:12.473321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5e68c-b31a-4061-8b0b-cf47e8aeb7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:12.473593 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5e68c-b31a-4061-8b0b-cf47e8aeb7d8 admin admin] PATH_INFO: `/v3/projects/735ffd0caa724744905c33edc9f0dd4e/users/ab1d5567f5fd46eeae07b57c25eef748/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:12.490917 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5e68c-b31a-4061-8b0b-cf47e8aeb7d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=735ffd0caa724744905c33edc9f0dd4e, user_id=ab1d5567f5fd46eeae07b57c25eef748, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:12.494125 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5e68c-b31a-4061-8b0b-cf47e8aeb7d8 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:12.670900 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1225/2440] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:22:12 2022] PUT /identity/v3/projects/735ffd0caa724744905c33edc9f0dd4e/users/ab1d5567f5fd46eeae07b57c25eef748/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:22:12.681041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ce573-c714-4446-b508-a6fe0b044755 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:12.681333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ce573-c714-4446-b508-a6fe0b044755 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:12.681614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ce573-c714-4446-b508-a6fe0b044755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:12.708130 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-df5ce573-c714-4446-b508-a6fe0b044755 None None] MFA Rules not processed for user `ab1d5567f5fd46eeae07b57c25eef748`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:12.709338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-df5ce573-c714-4446-b508-a6fe0b044755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:12.770688 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1216/2441] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:12.782646 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5e68c-b31a-4061-8b0b-cf47e8aeb7d8 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:12.784323 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-86f5e68c-b31a-4061-8b0b-cf47e8aeb7d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:12.856412 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5e68c-b31a-4061-8b0b-cf47e8aeb7d8 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:12.861894 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa509bc1-4aed-4bf9-a872-c1a1d122a6ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:12.862568 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa509bc1-4aed-4bf9-a872-c1a1d122a6ea service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:12.862856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa509bc1-4aed-4bf9-a872-c1a1d122a6ea service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:12.863134 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa509bc1-4aed-4bf9-a872-c1a1d122a6ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:12.870763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa509bc1-4aed-4bf9-a872-c1a1d122a6ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:12.872937 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa509bc1-4aed-4bf9-a872-c1a1d122a6ea service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:12.881620 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1226/2442] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:13.088216 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ce573-c714-4446-b508-a6fe0b044755 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:13.149203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ce573-c714-4446-b508-a6fe0b044755 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:13.154257 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f01300-91e3-42d1-8dba-f6e71daa0ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:13.155119 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f01300-91e3-42d1-8dba-f6e71daa0ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:13.155402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f01300-91e3-42d1-8dba-f6e71daa0ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:13.155678 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f01300-91e3-42d1-8dba-f6e71daa0ccc admin admin] PATH_INFO: `/v3/users/ab1d5567f5fd46eeae07b57c25eef748` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:13.156870 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f01300-91e3-42d1-8dba-f6e71daa0ccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab1d5567f5fd46eeae07b57c25eef748)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:13.158827 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f01300-91e3-42d1-8dba-f6e71daa0ccc admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:13.335514 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-20f01300-91e3-42d1-8dba-f6e71daa0ccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab1d5567f5fd46eeae07b57c25eef748'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:13.336143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-20f01300-91e3-42d1-8dba-f6e71daa0ccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab1d5567f5fd46eeae07b57c25eef748'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:13.347604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-20f01300-91e3-42d1-8dba-f6e71daa0ccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab1d5567f5fd46eeae07b57c25eef748'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:13.348247 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-20f01300-91e3-42d1-8dba-f6e71daa0ccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab1d5567f5fd46eeae07b57c25eef748'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:13.354880 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-20f01300-91e3-42d1-8dba-f6e71daa0ccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab1d5567f5fd46eeae07b57c25eef748'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:13.441808 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1217/2443] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:22:13 2022] DELETE /identity/v3/users/ab1d5567f5fd46eeae07b57c25eef748 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:13.446375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff4e0b-21de-4df1-999e-4b8b39d2ab6e None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:13.446674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff4e0b-21de-4df1-999e-4b8b39d2ab6e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:13.446958 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff4e0b-21de-4df1-999e-4b8b39d2ab6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:13.463820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-13ff4e0b-21de-4df1-999e-4b8b39d2ab6e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:13.465019 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-13ff4e0b-21de-4df1-999e-4b8b39d2ab6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:13.523133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1227/2444] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:13.533240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f01300-91e3-42d1-8dba-f6e71daa0ccc admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:13.534654 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-20f01300-91e3-42d1-8dba-f6e71daa0ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:13.591950 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f01300-91e3-42d1-8dba-f6e71daa0ccc admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:13.595540 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a26e3-cc93-4975-b368-10daaad8c9e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:13.596184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a26e3-cc93-4975-b368-10daaad8c9e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:13.596467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a26e3-cc93-4975-b368-10daaad8c9e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:13.596745 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a26e3-cc93-4975-b368-10daaad8c9e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:13.607046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a26e3-cc93-4975-b368-10daaad8c9e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:13.610290 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a26e3-cc93-4975-b368-10daaad8c9e0 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:13.618020 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1218/2445] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:14.061723 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff4e0b-21de-4df1-999e-4b8b39d2ab6e None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:14.063838 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-13ff4e0b-21de-4df1-999e-4b8b39d2ab6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:14.076067 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff4e0b-21de-4df1-999e-4b8b39d2ab6e None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:14.081008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d989618-1c9e-413d-913f-73ec3828bb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:14.081859 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d989618-1c9e-413d-913f-73ec3828bb93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:14.082157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d989618-1c9e-413d-913f-73ec3828bb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:14.082508 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d989618-1c9e-413d-913f-73ec3828bb93 admin admin] PATH_INFO: `/v3/projects/735ffd0caa724744905c33edc9f0dd4e` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:14.083653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d989618-1c9e-413d-913f-73ec3828bb93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=735ffd0caa724744905c33edc9f0dd4e)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:14.085556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d989618-1c9e-413d-913f-73ec3828bb93 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:14.218754 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-3d989618-1c9e-413d-913f-73ec3828bb93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '735ffd0caa724744905c33edc9f0dd4e', 'request_id': 'req-3d989618-1c9e-413d-913f-73ec3828bb93'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:14.293510 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-3d989618-1c9e-413d-913f-73ec3828bb93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '735ffd0caa724744905c33edc9f0dd4e', 'request_id': 'req-3d989618-1c9e-413d-913f-73ec3828bb93'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:14.299265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-3d989618-1c9e-413d-913f-73ec3828bb93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '735ffd0caa724744905c33edc9f0dd4e', 'request_id': 'req-3d989618-1c9e-413d-913f-73ec3828bb93'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:14.312138 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-3d989618-1c9e-413d-913f-73ec3828bb93 admin admin] The token cache is being invalidate because project 735ffd0caa724744905c33edc9f0dd4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:22:14.312422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-3d989618-1c9e-413d-913f-73ec3828bb93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:14.313073 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-3d989618-1c9e-413d-913f-73ec3828bb93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:14.314671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1228/2446] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:22:14 2022] DELETE /identity/v3/projects/735ffd0caa724744905c33edc9f0dd4e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:22:14.319513 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ca647-d591-40a9-83ac-e2e266e42464 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:14.319806 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ca647-d591-40a9-83ac-e2e266e42464 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:14.320083 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ca647-d591-40a9-83ac-e2e266e42464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:14.338449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-1f5ca647-d591-40a9-83ac-e2e266e42464 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:14.340307 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1f5ca647-d591-40a9-83ac-e2e266e42464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:14.397592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1219/2447] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:14.407146 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a182aa2-bdfe-4c50-aa88-4a69ee3d5c8b None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:14.410111 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a182aa2-bdfe-4c50-aa88-4a69ee3d5c8b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:14.410111 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a182aa2-bdfe-4c50-aa88-4a69ee3d5c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:14.426400 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-9a182aa2-bdfe-4c50-aa88-4a69ee3d5c8b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:14.427597 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-9a182aa2-bdfe-4c50-aa88-4a69ee3d5c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:14.436943 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1229/2448] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:22:14.441199 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ca647-d591-40a9-83ac-e2e266e42464 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:14.450960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ca647-d591-40a9-83ac-e2e266e42464 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:14.455811 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae3d72-7f4b-4867-893e-64b04b466bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:14.456456 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae3d72-7f4b-4867-893e-64b04b466bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:14.456740 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae3d72-7f4b-4867-893e-64b04b466bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:14.457015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae3d72-7f4b-4867-893e-64b04b466bb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:14.457775 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae3d72-7f4b-4867-893e-64b04b466bb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:22:14.458103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae3d72-7f4b-4867-893e-64b04b466bb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:14.459722 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae3d72-7f4b-4867-893e-64b04b466bb9 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:14.475401 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1220/2449] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:22:14.479696 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a182aa2-bdfe-4c50-aa88-4a69ee3d5c8b None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:14.489398 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a182aa2-bdfe-4c50-aa88-4a69ee3d5c8b None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:14.493940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27690b7-9902-4ba5-96ca-48e2c24866f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:14.494617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27690b7-9902-4ba5-96ca-48e2c24866f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:14.494901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27690b7-9902-4ba5-96ca-48e2c24866f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:14.495178 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27690b7-9902-4ba5-96ca-48e2c24866f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:14.496379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27690b7-9902-4ba5-96ca-48e2c24866f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:14.498292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27690b7-9902-4ba5-96ca-48e2c24866f2 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:14.597225 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1230/2450] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:22:14 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:22:14.601511 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae3d72-7f4b-4867-893e-64b04b466bb9 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:14.611697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae3d72-7f4b-4867-893e-64b04b466bb9 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:14.666782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc2939-9a0c-4e10-9a08-df6241d9db1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:14.667438 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc2939-9a0c-4e10-9a08-df6241d9db1f admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:14.667722 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc2939-9a0c-4e10-9a08-df6241d9db1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:14.668541 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc2939-9a0c-4e10-9a08-df6241d9db1f admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:14.669228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc2939-9a0c-4e10-9a08-df6241d9db1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:14.671165 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc2939-9a0c-4e10-9a08-df6241d9db1f admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:14.744535 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1221/2451] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:22:14 2022] POST /identity/v3/users => generated 283 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:22:14.748660 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27690b7-9902-4ba5-96ca-48e2c24866f2 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:14.758732 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27690b7-9902-4ba5-96ca-48e2c24866f2 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:14.763373 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a8f20-5a98-4bc7-ac37-11f43ea5991e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:14.764023 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a8f20-5a98-4bc7-ac37-11f43ea5991e admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:14.764305 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a8f20-5a98-4bc7-ac37-11f43ea5991e admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:14.764580 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a8f20-5a98-4bc7-ac37-11f43ea5991e admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:14.765312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a8f20-5a98-4bc7-ac37-11f43ea5991e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:14.766937 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a8f20-5a98-4bc7-ac37-11f43ea5991e admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:14.777046 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1231/2452] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:14.781660 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc2939-9a0c-4e10-9a08-df6241d9db1f admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:14.791516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc2939-9a0c-4e10-9a08-df6241d9db1f admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:14.796691 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80f081-ca41-474c-aa66-26bd6f2c0132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:14.797339 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80f081-ca41-474c-aa66-26bd6f2c0132 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:14.797616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80f081-ca41-474c-aa66-26bd6f2c0132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:14.797890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80f081-ca41-474c-aa66-26bd6f2c0132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:14.798646 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80f081-ca41-474c-aa66-26bd6f2c0132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:14.800212 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80f081-ca41-474c-aa66-26bd6f2c0132 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:14.810065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1222/2453] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:14.813955 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a8f20-5a98-4bc7-ac37-11f43ea5991e admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:14.823637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a8f20-5a98-4bc7-ac37-11f43ea5991e admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:14.828508 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fb7f9-715f-4fc6-919b-2ddef71fff18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:14.829367 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fb7f9-715f-4fc6-919b-2ddef71fff18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:14.829647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fb7f9-715f-4fc6-919b-2ddef71fff18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:14.829920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fb7f9-715f-4fc6-919b-2ddef71fff18 admin admin] PATH_INFO: `/v3/projects/5d927305b681418b836b83c5d99653fd/users/d4785d0700dd435ba5f0a3e200651fa1/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:14.844281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fb7f9-715f-4fc6-919b-2ddef71fff18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d927305b681418b836b83c5d99653fd, user_id=d4785d0700dd435ba5f0a3e200651fa1, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:14.846945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fb7f9-715f-4fc6-919b-2ddef71fff18 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:14.945152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1232/2454] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:22:14 2022] PUT /identity/v3/projects/5d927305b681418b836b83c5d99653fd/users/d4785d0700dd435ba5f0a3e200651fa1/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:14.956360 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6b73d-80ef-4c93-a25f-beef990becfb None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:14.956650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6b73d-80ef-4c93-a25f-beef990becfb None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:14.956926 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6b73d-80ef-4c93-a25f-beef990becfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:14.983197 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-43f6b73d-80ef-4c93-a25f-beef990becfb None None] MFA Rules not processed for user `d4785d0700dd435ba5f0a3e200651fa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:14.984401 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-43f6b73d-80ef-4c93-a25f-beef990becfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:15.048890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1223/2455] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:15.057041 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fb7f9-715f-4fc6-919b-2ddef71fff18 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:15.119349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fb7f9-715f-4fc6-919b-2ddef71fff18 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:15.124891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44bf5ff-bb37-4730-8fb2-4c84caf7a583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:15.125540 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bf5ff-bb37-4730-8fb2-4c84caf7a583 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:15.125825 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bf5ff-bb37-4730-8fb2-4c84caf7a583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:15.126126 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bf5ff-bb37-4730-8fb2-4c84caf7a583 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:15.126830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44bf5ff-bb37-4730-8fb2-4c84caf7a583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:15.128712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44bf5ff-bb37-4730-8fb2-4c84caf7a583 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:15.215320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1233/2456] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:22:15 2022] POST /identity/v3/projects => generated 365 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:22:15.219383 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6b73d-80ef-4c93-a25f-beef990becfb None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:15.229858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6b73d-80ef-4c93-a25f-beef990becfb None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:15.283765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccdc43-7400-4e12-b612-91c35d895c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:15.284419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccdc43-7400-4e12-b612-91c35d895c3d admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:15.284698 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccdc43-7400-4e12-b612-91c35d895c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:15.284972 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccdc43-7400-4e12-b612-91c35d895c3d admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:15.285655 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccdc43-7400-4e12-b612-91c35d895c3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:15.287577 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccdc43-7400-4e12-b612-91c35d895c3d admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:15.402424 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1224/2457] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:22:15 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:22:15.406469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44bf5ff-bb37-4730-8fb2-4c84caf7a583 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:15.416732 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44bf5ff-bb37-4730-8fb2-4c84caf7a583 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:15.421562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0858c4-89e3-44a8-82c6-7c94e8d3de86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:15.424145 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0858c4-89e3-44a8-82c6-7c94e8d3de86 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:15.424145 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0858c4-89e3-44a8-82c6-7c94e8d3de86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:15.424145 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0858c4-89e3-44a8-82c6-7c94e8d3de86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:15.424145 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0858c4-89e3-44a8-82c6-7c94e8d3de86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:15.425205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0858c4-89e3-44a8-82c6-7c94e8d3de86 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:15.436678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1234/2458] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:15.440844 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccdc43-7400-4e12-b612-91c35d895c3d admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:15.451407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccdc43-7400-4e12-b612-91c35d895c3d admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:15.456279 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03d2ee-51b7-495d-9035-38f7d57a0c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:15.457147 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03d2ee-51b7-495d-9035-38f7d57a0c1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:15.457432 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03d2ee-51b7-495d-9035-38f7d57a0c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:15.457709 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03d2ee-51b7-495d-9035-38f7d57a0c1a admin admin] PATH_INFO: `/v3/projects/dc100911056342f3b77499d8e2dec08a/users/fae82dd10c774051a8d84409a47bbd69/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:15.471874 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03d2ee-51b7-495d-9035-38f7d57a0c1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc100911056342f3b77499d8e2dec08a, user_id=fae82dd10c774051a8d84409a47bbd69, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:15.474559 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03d2ee-51b7-495d-9035-38f7d57a0c1a admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:15.544671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1225/2459] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:22:15 2022] PUT /identity/v3/projects/dc100911056342f3b77499d8e2dec08a/users/fae82dd10c774051a8d84409a47bbd69/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:15.548625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0858c4-89e3-44a8-82c6-7c94e8d3de86 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:15.559036 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0858c4-89e3-44a8-82c6-7c94e8d3de86 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:15.612463 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22395f-855e-4926-a1ed-d8bc67c54223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:15.613104 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22395f-855e-4926-a1ed-d8bc67c54223 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:15.613388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22395f-855e-4926-a1ed-d8bc67c54223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:15.613665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22395f-855e-4926-a1ed-d8bc67c54223 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:15.614431 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e22395f-855e-4926-a1ed-d8bc67c54223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:15.616475 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e22395f-855e-4926-a1ed-d8bc67c54223 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:15.626501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1235/2460] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:15.630553 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03d2ee-51b7-495d-9035-38f7d57a0c1a admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:15.640688 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03d2ee-51b7-495d-9035-38f7d57a0c1a admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:15.645207 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5edb9-b082-45f7-98d0-db7d2e672773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:15.646071 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5edb9-b082-45f7-98d0-db7d2e672773 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:15.646377 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5edb9-b082-45f7-98d0-db7d2e672773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:15.646647 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5edb9-b082-45f7-98d0-db7d2e672773 admin admin] PATH_INFO: `/v3/projects/dc100911056342f3b77499d8e2dec08a/users/fae82dd10c774051a8d84409a47bbd69/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:15.648582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5edb9-b082-45f7-98d0-db7d2e672773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc100911056342f3b77499d8e2dec08a, user_id=fae82dd10c774051a8d84409a47bbd69, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:15.651218 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5edb9-b082-45f7-98d0-db7d2e672773 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:15.661227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1226/2461] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:22:15 2022] PUT /identity/v3/projects/dc100911056342f3b77499d8e2dec08a/users/fae82dd10c774051a8d84409a47bbd69/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:15.671774 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abf6a3-3203-447d-811d-25ce55706a3c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:15.672068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abf6a3-3203-447d-811d-25ce55706a3c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:15.672354 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abf6a3-3203-447d-811d-25ce55706a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:15.698282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-05abf6a3-3203-447d-811d-25ce55706a3c None None] MFA Rules not processed for user `fae82dd10c774051a8d84409a47bbd69`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:15.700051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-05abf6a3-3203-447d-811d-25ce55706a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:15.765198 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1236/2462] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:22:15.776311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5edb9-b082-45f7-98d0-db7d2e672773 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:15.778661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-51a5edb9-b082-45f7-98d0-db7d2e672773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:15.836879 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5edb9-b082-45f7-98d0-db7d2e672773 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:15.841250 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4b7eb-53a1-487f-b9c0-f8a7630c129a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:15.841888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4b7eb-53a1-487f-b9c0-f8a7630c129a service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:15.842211 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4b7eb-53a1-487f-b9c0-f8a7630c129a service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:15.842487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4b7eb-53a1-487f-b9c0-f8a7630c129a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:15.851184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4b7eb-53a1-487f-b9c0-f8a7630c129a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:15.852596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4b7eb-53a1-487f-b9c0-f8a7630c129a service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:15.861784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1227/2463] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:16.064609 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abf6a3-3203-447d-811d-25ce55706a3c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:16.073973 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abf6a3-3203-447d-811d-25ce55706a3c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:16.078013 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9364d4-f0ff-4ff6-8203-3b8e97fe3d26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:16.078672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9364d4-f0ff-4ff6-8203-3b8e97fe3d26 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:16.078952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9364d4-f0ff-4ff6-8203-3b8e97fe3d26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:16.079227 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9364d4-f0ff-4ff6-8203-3b8e97fe3d26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:16.134033 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9364d4-f0ff-4ff6-8203-3b8e97fe3d26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:16.135491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9364d4-f0ff-4ff6-8203-3b8e97fe3d26 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:16.144879 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1237/2464] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:19.450796 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4b7eb-53a1-487f-b9c0-f8a7630c129a service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:19.510639 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4b7eb-53a1-487f-b9c0-f8a7630c129a service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:19.515089 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac00fc-fbf0-4c52-8b71-30330a4d83b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:19.515935 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac00fc-fbf0-4c52-8b71-30330a4d83b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:19.516211 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac00fc-fbf0-4c52-8b71-30330a4d83b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:19.516480 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac00fc-fbf0-4c52-8b71-30330a4d83b8 admin admin] PATH_INFO: `/v3/users/d4785d0700dd435ba5f0a3e200651fa1` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:19.517633 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac00fc-fbf0-4c52-8b71-30330a4d83b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4785d0700dd435ba5f0a3e200651fa1)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:19.519561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac00fc-fbf0-4c52-8b71-30330a4d83b8 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:19.724519 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-91ac00fc-fbf0-4c52-8b71-30330a4d83b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4785d0700dd435ba5f0a3e200651fa1'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:19.725161 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-91ac00fc-fbf0-4c52-8b71-30330a4d83b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4785d0700dd435ba5f0a3e200651fa1'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:19.736444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-91ac00fc-fbf0-4c52-8b71-30330a4d83b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4785d0700dd435ba5f0a3e200651fa1'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:19.737074 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-91ac00fc-fbf0-4c52-8b71-30330a4d83b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4785d0700dd435ba5f0a3e200651fa1'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:19.743604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-91ac00fc-fbf0-4c52-8b71-30330a4d83b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4785d0700dd435ba5f0a3e200651fa1'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:19.818667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1228/2465] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:22:19 2022] DELETE /identity/v3/users/d4785d0700dd435ba5f0a3e200651fa1 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:19.823153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4748234-17de-4266-acd9-0007dc324354 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:19.823621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4748234-17de-4266-acd9-0007dc324354 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:19.823719 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4748234-17de-4266-acd9-0007dc324354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:19.841864 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-b4748234-17de-4266-acd9-0007dc324354 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:19.843098 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b4748234-17de-4266-acd9-0007dc324354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:19.899423 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1238/2466] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:19.909115 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac00fc-fbf0-4c52-8b71-30330a4d83b8 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:19.910522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-91ac00fc-fbf0-4c52-8b71-30330a4d83b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:19.966516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac00fc-fbf0-4c52-8b71-30330a4d83b8 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:19.970068 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3b554-c493-4306-abb6-c647341303e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:19.970738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3b554-c493-4306-abb6-c647341303e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:19.971024 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3b554-c493-4306-abb6-c647341303e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:19.971302 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3b554-c493-4306-abb6-c647341303e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:19.980088 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3b554-c493-4306-abb6-c647341303e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:19.982051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3b554-c493-4306-abb6-c647341303e2 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:19.991263 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1229/2467] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:20.392185 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4748234-17de-4266-acd9-0007dc324354 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:20.393664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b4748234-17de-4266-acd9-0007dc324354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:20.407968 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4748234-17de-4266-acd9-0007dc324354 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:20.413438 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57845b54-525b-4e85-a6a0-04ebbaabba62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:20.414327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57845b54-525b-4e85-a6a0-04ebbaabba62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:20.414614 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57845b54-525b-4e85-a6a0-04ebbaabba62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:20.414910 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57845b54-525b-4e85-a6a0-04ebbaabba62 admin admin] PATH_INFO: `/v3/projects/5d927305b681418b836b83c5d99653fd` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:20.416125 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57845b54-525b-4e85-a6a0-04ebbaabba62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d927305b681418b836b83c5d99653fd)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:20.418653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57845b54-525b-4e85-a6a0-04ebbaabba62 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:20.532576 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-57845b54-525b-4e85-a6a0-04ebbaabba62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d927305b681418b836b83c5d99653fd', 'request_id': 'req-57845b54-525b-4e85-a6a0-04ebbaabba62'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:20.590952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-57845b54-525b-4e85-a6a0-04ebbaabba62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d927305b681418b836b83c5d99653fd', 'request_id': 'req-57845b54-525b-4e85-a6a0-04ebbaabba62'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:20.596709 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-57845b54-525b-4e85-a6a0-04ebbaabba62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d927305b681418b836b83c5d99653fd', 'request_id': 'req-57845b54-525b-4e85-a6a0-04ebbaabba62'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:20.609050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-57845b54-525b-4e85-a6a0-04ebbaabba62 admin admin] The token cache is being invalidate because project 5d927305b681418b836b83c5d99653fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:22:20.609342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-57845b54-525b-4e85-a6a0-04ebbaabba62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:20.609990 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-57845b54-525b-4e85-a6a0-04ebbaabba62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:20.611575 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1239/2468] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:22:20 2022] DELETE /identity/v3/projects/5d927305b681418b836b83c5d99653fd => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:20.615543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3b554-c493-4306-abb6-c647341303e2 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:20.625926 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3b554-c493-4306-abb6-c647341303e2 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:20.681334 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b08f1-ed19-4915-b32e-3a662f26d5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:20.682209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b08f1-ed19-4915-b32e-3a662f26d5ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:20.682596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b08f1-ed19-4915-b32e-3a662f26d5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:20.682878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b08f1-ed19-4915-b32e-3a662f26d5ea admin admin] PATH_INFO: `/v3/users/fae82dd10c774051a8d84409a47bbd69` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:20.684696 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b08f1-ed19-4915-b32e-3a662f26d5ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=fae82dd10c774051a8d84409a47bbd69)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:20.686614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b08f1-ed19-4915-b32e-3a662f26d5ea admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:20.827018 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-5f6b08f1-ed19-4915-b32e-3a662f26d5ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fae82dd10c774051a8d84409a47bbd69'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:20.827942 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-5f6b08f1-ed19-4915-b32e-3a662f26d5ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fae82dd10c774051a8d84409a47bbd69'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:20.839371 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-5f6b08f1-ed19-4915-b32e-3a662f26d5ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fae82dd10c774051a8d84409a47bbd69'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:20.840622 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-5f6b08f1-ed19-4915-b32e-3a662f26d5ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fae82dd10c774051a8d84409a47bbd69'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:20.847136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-5f6b08f1-ed19-4915-b32e-3a662f26d5ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fae82dd10c774051a8d84409a47bbd69'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:20.921595 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1230/2469] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:22:20 2022] DELETE /identity/v3/users/fae82dd10c774051a8d84409a47bbd69 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:21.334559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57845b54-525b-4e85-a6a0-04ebbaabba62 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:21.342325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-57845b54-525b-4e85-a6a0-04ebbaabba62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:21.349413 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57845b54-525b-4e85-a6a0-04ebbaabba62 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:21.410691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066af78-da58-4791-92d3-3adb8a903d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:21.411544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066af78-da58-4791-92d3-3adb8a903d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:21.411830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066af78-da58-4791-92d3-3adb8a903d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:21.412125 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066af78-da58-4791-92d3-3adb8a903d4a admin admin] PATH_INFO: `/v3/projects/dc100911056342f3b77499d8e2dec08a` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:21.413283 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066af78-da58-4791-92d3-3adb8a903d4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc100911056342f3b77499d8e2dec08a)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:21.415214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066af78-da58-4791-92d3-3adb8a903d4a admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:21.526469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e066af78-da58-4791-92d3-3adb8a903d4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc100911056342f3b77499d8e2dec08a', 'request_id': 'req-e066af78-da58-4791-92d3-3adb8a903d4a'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:21.596154 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e066af78-da58-4791-92d3-3adb8a903d4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc100911056342f3b77499d8e2dec08a', 'request_id': 'req-e066af78-da58-4791-92d3-3adb8a903d4a'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:21.601564 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e066af78-da58-4791-92d3-3adb8a903d4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc100911056342f3b77499d8e2dec08a', 'request_id': 'req-e066af78-da58-4791-92d3-3adb8a903d4a'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:21.612864 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e066af78-da58-4791-92d3-3adb8a903d4a admin admin] The token cache is being invalidate because project dc100911056342f3b77499d8e2dec08a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:22:21.613154 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e066af78-da58-4791-92d3-3adb8a903d4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:21.613806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e066af78-da58-4791-92d3-3adb8a903d4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:21.615367 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1240/2470] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:22:21 2022] DELETE /identity/v3/projects/dc100911056342f3b77499d8e2dec08a => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:21.620209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394a40e-624e-4d15-8d36-a6fc130bce2d None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:21.620700 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394a40e-624e-4d15-8d36-a6fc130bce2d None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:21.620822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394a40e-624e-4d15-8d36-a6fc130bce2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:21.638599 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-7394a40e-624e-4d15-8d36-a6fc130bce2d None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:21.639782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-7394a40e-624e-4d15-8d36-a6fc130bce2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:21.698527 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1231/2471] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:21.713328 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea25526-0d21-4ac8-9914-827f25c4898c None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:21.713613 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea25526-0d21-4ac8-9914-827f25c4898c None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:21.713892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea25526-0d21-4ac8-9914-827f25c4898c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:21.732949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-7ea25526-0d21-4ac8-9914-827f25c4898c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:21.734169 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7ea25526-0d21-4ac8-9914-827f25c4898c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:21.741987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1241/2472] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:21.746924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394a40e-624e-4d15-8d36-a6fc130bce2d None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:21.756600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394a40e-624e-4d15-8d36-a6fc130bce2d None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:21.761251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81585533-179b-402d-8ea9-f2408b5be510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:21.761923 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81585533-179b-402d-8ea9-f2408b5be510 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:21.762242 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81585533-179b-402d-8ea9-f2408b5be510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:21.762524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81585533-179b-402d-8ea9-f2408b5be510 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:21.763253 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81585533-179b-402d-8ea9-f2408b5be510 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:22:21.763557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81585533-179b-402d-8ea9-f2408b5be510 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:21.765149 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81585533-179b-402d-8ea9-f2408b5be510 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:21.776570 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1232/2473] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:22:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:22:21.780655 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea25526-0d21-4ac8-9914-827f25c4898c None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:21.789675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea25526-0d21-4ac8-9914-827f25c4898c None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:21.794864 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210a930-7672-4028-a832-876fc24c9cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:21.795502 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210a930-7672-4028-a832-876fc24c9cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:21.795783 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210a930-7672-4028-a832-876fc24c9cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:21.796056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210a930-7672-4028-a832-876fc24c9cb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:21.796758 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210a930-7672-4028-a832-876fc24c9cb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:21.798671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210a930-7672-4028-a832-876fc24c9cb6 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:21.872502 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1242/2474] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:22:21 2022] POST /identity/v3/projects => generated 369 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:22:21.876632 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81585533-179b-402d-8ea9-f2408b5be510 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:21.887218 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81585533-179b-402d-8ea9-f2408b5be510 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:21.941079 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd2fce-24a9-4649-8750-a7df20d1ea40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:21.941829 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd2fce-24a9-4649-8750-a7df20d1ea40 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:21.942143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd2fce-24a9-4649-8750-a7df20d1ea40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:21.942422 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd2fce-24a9-4649-8750-a7df20d1ea40 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:21.943108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd2fce-24a9-4649-8750-a7df20d1ea40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:21.945025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd2fce-24a9-4649-8750-a7df20d1ea40 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:22.013808 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1233/2475] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:22:21 2022] POST /identity/v3/users => generated 285 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:22:22.017730 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210a930-7672-4028-a832-876fc24c9cb6 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:22.028538 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210a930-7672-4028-a832-876fc24c9cb6 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:22.033241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6e898-2c75-413c-b1fb-40bb8c9a1502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:22.033894 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6e898-2c75-413c-b1fb-40bb8c9a1502 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:22.034210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6e898-2c75-413c-b1fb-40bb8c9a1502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:22.034489 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6e898-2c75-413c-b1fb-40bb8c9a1502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:22.035256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6e898-2c75-413c-b1fb-40bb8c9a1502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:22.036906 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6e898-2c75-413c-b1fb-40bb8c9a1502 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:22.047768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1243/2476] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:22.060580 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd2fce-24a9-4649-8750-a7df20d1ea40 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:22.070775 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd2fce-24a9-4649-8750-a7df20d1ea40 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:22.076388 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e33446-715e-438a-bd26-e098e67ac8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:22.077069 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e33446-715e-438a-bd26-e098e67ac8f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:22.077350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e33446-715e-438a-bd26-e098e67ac8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:22.077878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e33446-715e-438a-bd26-e098e67ac8f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:22.078630 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e33446-715e-438a-bd26-e098e67ac8f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:22.080222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e33446-715e-438a-bd26-e098e67ac8f2 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:22.090231 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1234/2477] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:22.094570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6e898-2c75-413c-b1fb-40bb8c9a1502 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:22.104418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6e898-2c75-413c-b1fb-40bb8c9a1502 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:22.109050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88fa46-0cbd-4147-8e05-6f4d33fff84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:22.109910 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88fa46-0cbd-4147-8e05-6f4d33fff84f admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:22.110249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88fa46-0cbd-4147-8e05-6f4d33fff84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:22.110528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88fa46-0cbd-4147-8e05-6f4d33fff84f admin admin] PATH_INFO: `/v3/projects/dfd15852cc7d444ea872fb09b1227097/users/3695fda7ad31472c9ed34c634c722d68/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:22.124961 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88fa46-0cbd-4147-8e05-6f4d33fff84f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfd15852cc7d444ea872fb09b1227097, user_id=3695fda7ad31472c9ed34c634c722d68, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:22.127669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88fa46-0cbd-4147-8e05-6f4d33fff84f admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:22.178808 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1244/2478] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:22:22 2022] PUT /identity/v3/projects/dfd15852cc7d444ea872fb09b1227097/users/3695fda7ad31472c9ed34c634c722d68/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:22.188879 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3b5ce-09ee-4ac9-8957-8c974f608b5c None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:22.189832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3b5ce-09ee-4ac9-8957-8c974f608b5c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:22.190166 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3b5ce-09ee-4ac9-8957-8c974f608b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:22.217343 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-a2e3b5ce-09ee-4ac9-8957-8c974f608b5c None None] MFA Rules not processed for user `3695fda7ad31472c9ed34c634c722d68`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:22.218601 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a2e3b5ce-09ee-4ac9-8957-8c974f608b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:22.279566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1235/2479] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:22 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:22:22.284021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88fa46-0cbd-4147-8e05-6f4d33fff84f admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:22.343766 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88fa46-0cbd-4147-8e05-6f4d33fff84f admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:22.348463 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df7465-8a79-4f49-ba85-3cf13c1863a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:22.349109 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df7465-8a79-4f49-ba85-3cf13c1863a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:22.349397 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df7465-8a79-4f49-ba85-3cf13c1863a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:22.349676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df7465-8a79-4f49-ba85-3cf13c1863a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:22.350427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df7465-8a79-4f49-ba85-3cf13c1863a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:22.352318 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df7465-8a79-4f49-ba85-3cf13c1863a4 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:22.443992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1245/2480] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:22:22 2022] POST /identity/v3/projects => generated 367 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:22:22.448001 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3b5ce-09ee-4ac9-8957-8c974f608b5c None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:22.459327 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3b5ce-09ee-4ac9-8957-8c974f608b5c None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:22.514740 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c686f-5cc7-4293-8510-ca496b9c69d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:22.515399 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c686f-5cc7-4293-8510-ca496b9c69d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:22.515682 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c686f-5cc7-4293-8510-ca496b9c69d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:22.515961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c686f-5cc7-4293-8510-ca496b9c69d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:22.516642 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c686f-5cc7-4293-8510-ca496b9c69d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:22.518573 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c686f-5cc7-4293-8510-ca496b9c69d4 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:22.619940 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1236/2481] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:22:22 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:22:22.624547 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df7465-8a79-4f49-ba85-3cf13c1863a4 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:22.634870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df7465-8a79-4f49-ba85-3cf13c1863a4 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:22.639447 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde4690-d812-40a3-bde4-a044b515c346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:22.640097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde4690-d812-40a3-bde4-a044b515c346 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:22.640380 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde4690-d812-40a3-bde4-a044b515c346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:22.640657 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde4690-d812-40a3-bde4-a044b515c346 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:22.641418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde4690-d812-40a3-bde4-a044b515c346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:22.643029 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde4690-d812-40a3-bde4-a044b515c346 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:22.653370 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1246/2482] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:22.658344 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c686f-5cc7-4293-8510-ca496b9c69d4 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:22.668671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c686f-5cc7-4293-8510-ca496b9c69d4 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:22.673282 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7434809-83c5-4f32-866e-5979b4f13456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:22.674178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7434809-83c5-4f32-866e-5979b4f13456 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:22.674462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7434809-83c5-4f32-866e-5979b4f13456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:22.674737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7434809-83c5-4f32-866e-5979b4f13456 admin admin] PATH_INFO: `/v3/projects/76d84e29cc1e4a6fa5583fb881926825/users/667cdb9b12a04dddb9b153ca50c464fa/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:22.689480 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7434809-83c5-4f32-866e-5979b4f13456 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d84e29cc1e4a6fa5583fb881926825, user_id=667cdb9b12a04dddb9b153ca50c464fa, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:22.692162 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7434809-83c5-4f32-866e-5979b4f13456 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:22.796591 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1237/2483] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:22:22 2022] PUT /identity/v3/projects/76d84e29cc1e4a6fa5583fb881926825/users/667cdb9b12a04dddb9b153ca50c464fa/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:22:22.800605 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde4690-d812-40a3-bde4-a044b515c346 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:22.811532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde4690-d812-40a3-bde4-a044b515c346 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:22.865499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c463be6-a1df-4102-b611-0387370e4e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:22.866183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c463be6-a1df-4102-b611-0387370e4e0c admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:22.866470 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c463be6-a1df-4102-b611-0387370e4e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:22.866748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c463be6-a1df-4102-b611-0387370e4e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:22.867477 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c463be6-a1df-4102-b611-0387370e4e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:22.869063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c463be6-a1df-4102-b611-0387370e4e0c admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:22.879248 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1247/2484] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:22:22.883388 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7434809-83c5-4f32-866e-5979b4f13456 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:22.894341 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7434809-83c5-4f32-866e-5979b4f13456 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:22.898955 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced5210-796b-478a-a2b8-e4c35afe4fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:22.899824 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced5210-796b-478a-a2b8-e4c35afe4fd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:22.900126 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced5210-796b-478a-a2b8-e4c35afe4fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:22.900381 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced5210-796b-478a-a2b8-e4c35afe4fd9 admin admin] PATH_INFO: `/v3/projects/76d84e29cc1e4a6fa5583fb881926825/users/667cdb9b12a04dddb9b153ca50c464fa/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:22.902551 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced5210-796b-478a-a2b8-e4c35afe4fd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d84e29cc1e4a6fa5583fb881926825, user_id=667cdb9b12a04dddb9b153ca50c464fa, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:22.905203 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced5210-796b-478a-a2b8-e4c35afe4fd9 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:22.915947 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1238/2485] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:22:22 2022] PUT /identity/v3/projects/76d84e29cc1e4a6fa5583fb881926825/users/667cdb9b12a04dddb9b153ca50c464fa/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:22.928139 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12868959-bc1b-443b-991e-3f13a78f91f8 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:22.928414 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12868959-bc1b-443b-991e-3f13a78f91f8 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:22.928700 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12868959-bc1b-443b-991e-3f13a78f91f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:22.955923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-12868959-bc1b-443b-991e-3f13a78f91f8 None None] MFA Rules not processed for user `667cdb9b12a04dddb9b153ca50c464fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:22.957212 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-12868959-bc1b-443b-991e-3f13a78f91f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:23.021329 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1248/2486] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:22 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:23.034227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced5210-796b-478a-a2b8-e4c35afe4fd9 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:23.035895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0ced5210-796b-478a-a2b8-e4c35afe4fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:23.093707 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced5210-796b-478a-a2b8-e4c35afe4fd9 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:23.097390 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a79dd-44c0-4683-a90b-685833687678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:23.098031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a79dd-44c0-4683-a90b-685833687678 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:23.099160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a79dd-44c0-4683-a90b-685833687678 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:23.099160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a79dd-44c0-4683-a90b-685833687678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:23.115193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a79dd-44c0-4683-a90b-685833687678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:23.116744 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a79dd-44c0-4683-a90b-685833687678 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:23.129275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1239/2487] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:23 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:23.258259 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12868959-bc1b-443b-991e-3f13a78f91f8 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:23.259663 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-12868959-bc1b-443b-991e-3f13a78f91f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:23.271806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12868959-bc1b-443b-991e-3f13a78f91f8 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:23.275388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7ff0e-cb1a-499b-b585-49451181aa99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:23.276022 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7ff0e-cb1a-499b-b585-49451181aa99 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:23.276313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7ff0e-cb1a-499b-b585-49451181aa99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:23.276593 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7ff0e-cb1a-499b-b585-49451181aa99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:23.331463 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7ff0e-cb1a-499b-b585-49451181aa99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:23.332889 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7ff0e-cb1a-499b-b585-49451181aa99 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:23.342166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1249/2488] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:23 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:23.814057 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a79dd-44c0-4683-a90b-685833687678 service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:23.815508 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-322a79dd-44c0-4683-a90b-685833687678 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:23.887018 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a79dd-44c0-4683-a90b-685833687678 service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:23.893002 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f75b8a-40d8-4b3d-98eb-c3bceb868492 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:23.893655 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f75b8a-40d8-4b3d-98eb-c3bceb868492 service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:23.893944 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f75b8a-40d8-4b3d-98eb-c3bceb868492 service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:23.894266 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f75b8a-40d8-4b3d-98eb-c3bceb868492 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:23.896078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-12f75b8a-40d8-4b3d-98eb-c3bceb868492 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:23.955787 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f75b8a-40d8-4b3d-98eb-c3bceb868492 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:23.957372 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f75b8a-40d8-4b3d-98eb-c3bceb868492 service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:23.966407 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1240/2489] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:28.016492 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7ff0e-cb1a-499b-b585-49451181aa99 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:28.079016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7ff0e-cb1a-499b-b585-49451181aa99 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:28.083564 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015fde2-6b27-4ee1-8845-61ce25cfe068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:28.084421 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015fde2-6b27-4ee1-8845-61ce25cfe068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:28.084701 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015fde2-6b27-4ee1-8845-61ce25cfe068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:28.084975 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015fde2-6b27-4ee1-8845-61ce25cfe068 admin admin] PATH_INFO: `/v3/users/3695fda7ad31472c9ed34c634c722d68` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:28.086213 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015fde2-6b27-4ee1-8845-61ce25cfe068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3695fda7ad31472c9ed34c634c722d68)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:28.088125 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015fde2-6b27-4ee1-8845-61ce25cfe068 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:28.214795 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8015fde2-6b27-4ee1-8845-61ce25cfe068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3695fda7ad31472c9ed34c634c722d68'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:28.215453 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8015fde2-6b27-4ee1-8845-61ce25cfe068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3695fda7ad31472c9ed34c634c722d68'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:28.216061 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8015fde2-6b27-4ee1-8845-61ce25cfe068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3695fda7ad31472c9ed34c634c722d68'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:28.232865 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8015fde2-6b27-4ee1-8845-61ce25cfe068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3695fda7ad31472c9ed34c634c722d68'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:28.244638 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8015fde2-6b27-4ee1-8845-61ce25cfe068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3695fda7ad31472c9ed34c634c722d68'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:28.331208 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1250/2490] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:22:28 2022] DELETE /identity/v3/users/3695fda7ad31472c9ed34c634c722d68 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:28.335688 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ef631-3267-401f-854f-ade4ee721a82 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:28.336190 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ef631-3267-401f-854f-ade4ee721a82 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:28.336305 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ef631-3267-401f-854f-ade4ee721a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:28.353994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-162ef631-3267-401f-854f-ade4ee721a82 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:28.355258 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-162ef631-3267-401f-854f-ade4ee721a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:28.415418 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1241/2491] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:28.424941 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015fde2-6b27-4ee1-8845-61ce25cfe068 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:28.426558 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-8015fde2-6b27-4ee1-8845-61ce25cfe068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:28.490168 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015fde2-6b27-4ee1-8845-61ce25cfe068 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:28.493779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39439269-be2b-49c0-b137-c30a6869c666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:28.494460 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39439269-be2b-49c0-b137-c30a6869c666 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:28.494746 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39439269-be2b-49c0-b137-c30a6869c666 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:28.495022 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39439269-be2b-49c0-b137-c30a6869c666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:28.504189 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39439269-be2b-49c0-b137-c30a6869c666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:28.505537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39439269-be2b-49c0-b137-c30a6869c666 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:28.515219 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1251/2492] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:28.895477 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ef631-3267-401f-854f-ade4ee721a82 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:28.896874 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-162ef631-3267-401f-854f-ade4ee721a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:28.914541 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ef631-3267-401f-854f-ade4ee721a82 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:28.919919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f3f51-1d04-475f-b1d3-e799ee2f43e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:28.920883 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f3f51-1d04-475f-b1d3-e799ee2f43e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:28.921059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f3f51-1d04-475f-b1d3-e799ee2f43e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:28.921389 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f3f51-1d04-475f-b1d3-e799ee2f43e0 admin admin] PATH_INFO: `/v3/projects/dfd15852cc7d444ea872fb09b1227097` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:28.922544 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f3f51-1d04-475f-b1d3-e799ee2f43e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfd15852cc7d444ea872fb09b1227097)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:28.924448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f3f51-1d04-475f-b1d3-e799ee2f43e0 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:29.036116 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-986f3f51-1d04-475f-b1d3-e799ee2f43e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfd15852cc7d444ea872fb09b1227097', 'request_id': 'req-986f3f51-1d04-475f-b1d3-e799ee2f43e0'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:29.104889 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-986f3f51-1d04-475f-b1d3-e799ee2f43e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfd15852cc7d444ea872fb09b1227097', 'request_id': 'req-986f3f51-1d04-475f-b1d3-e799ee2f43e0'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:29.111095 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-986f3f51-1d04-475f-b1d3-e799ee2f43e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfd15852cc7d444ea872fb09b1227097', 'request_id': 'req-986f3f51-1d04-475f-b1d3-e799ee2f43e0'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:29.121881 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-986f3f51-1d04-475f-b1d3-e799ee2f43e0 admin admin] The token cache is being invalidate because project dfd15852cc7d444ea872fb09b1227097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:22:29.122793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-986f3f51-1d04-475f-b1d3-e799ee2f43e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:29.130354 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-986f3f51-1d04-475f-b1d3-e799ee2f43e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:29.130354 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1242/2493] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:22:28 2022] DELETE /identity/v3/projects/dfd15852cc7d444ea872fb09b1227097 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:29.130609 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39439269-be2b-49c0-b137-c30a6869c666 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:29.138504 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39439269-be2b-49c0-b137-c30a6869c666 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:29.193410 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083a1bc-83ab-4cdc-bcc8-8288e7ea7574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:29.195023 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083a1bc-83ab-4cdc-bcc8-8288e7ea7574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:29.195307 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083a1bc-83ab-4cdc-bcc8-8288e7ea7574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:29.195578 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083a1bc-83ab-4cdc-bcc8-8288e7ea7574 admin admin] PATH_INFO: `/v3/users/667cdb9b12a04dddb9b153ca50c464fa` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:29.196811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083a1bc-83ab-4cdc-bcc8-8288e7ea7574 admin admin] RBAC: Authorizing `identity:delete_user(user_id=667cdb9b12a04dddb9b153ca50c464fa)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:29.198776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083a1bc-83ab-4cdc-bcc8-8288e7ea7574 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:29.403519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8083a1bc-83ab-4cdc-bcc8-8288e7ea7574 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '667cdb9b12a04dddb9b153ca50c464fa'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:29.404091 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8083a1bc-83ab-4cdc-bcc8-8288e7ea7574 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '667cdb9b12a04dddb9b153ca50c464fa'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:29.404699 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8083a1bc-83ab-4cdc-bcc8-8288e7ea7574 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '667cdb9b12a04dddb9b153ca50c464fa'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:29.416223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8083a1bc-83ab-4cdc-bcc8-8288e7ea7574 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '667cdb9b12a04dddb9b153ca50c464fa'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:29.424158 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8083a1bc-83ab-4cdc-bcc8-8288e7ea7574 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '667cdb9b12a04dddb9b153ca50c464fa'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:29.477384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1252/2494] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:22:29 2022] DELETE /identity/v3/users/667cdb9b12a04dddb9b153ca50c464fa => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:29.850120 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f3f51-1d04-475f-b1d3-e799ee2f43e0 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:29.851604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-986f3f51-1d04-475f-b1d3-e799ee2f43e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:29.865427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f3f51-1d04-475f-b1d3-e799ee2f43e0 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:29.921481 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0096f-a47e-4100-8e22-8ee3a8a59fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:29.922307 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0096f-a47e-4100-8e22-8ee3a8a59fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:29.922591 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0096f-a47e-4100-8e22-8ee3a8a59fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:29.922870 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0096f-a47e-4100-8e22-8ee3a8a59fb9 admin admin] PATH_INFO: `/v3/projects/76d84e29cc1e4a6fa5583fb881926825` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:29.924052 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f0096f-a47e-4100-8e22-8ee3a8a59fb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d84e29cc1e4a6fa5583fb881926825)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:29.925961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f0096f-a47e-4100-8e22-8ee3a8a59fb9 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:30.048041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-53f0096f-a47e-4100-8e22-8ee3a8a59fb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d84e29cc1e4a6fa5583fb881926825', 'request_id': 'req-53f0096f-a47e-4100-8e22-8ee3a8a59fb9'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:30.123547 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-53f0096f-a47e-4100-8e22-8ee3a8a59fb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d84e29cc1e4a6fa5583fb881926825', 'request_id': 'req-53f0096f-a47e-4100-8e22-8ee3a8a59fb9'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:30.129651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-53f0096f-a47e-4100-8e22-8ee3a8a59fb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d84e29cc1e4a6fa5583fb881926825', 'request_id': 'req-53f0096f-a47e-4100-8e22-8ee3a8a59fb9'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:30.145064 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-53f0096f-a47e-4100-8e22-8ee3a8a59fb9 admin admin] The token cache is being invalidate because project 76d84e29cc1e4a6fa5583fb881926825 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:22:30.145350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-53f0096f-a47e-4100-8e22-8ee3a8a59fb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:30.146038 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-53f0096f-a47e-4100-8e22-8ee3a8a59fb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:30.147604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1243/2495] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:22:29 2022] DELETE /identity/v3/projects/76d84e29cc1e4a6fa5583fb881926825 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:30.152611 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6193a-d501-404d-bf78-3585c7692a2b None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:30.153154 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6193a-d501-404d-bf78-3585c7692a2b None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:30.153154 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6193a-d501-404d-bf78-3585c7692a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:30.171990 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-51c6193a-d501-404d-bf78-3585c7692a2b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:30.173174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-51c6193a-d501-404d-bf78-3585c7692a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:30.233131 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1253/2496] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:30.240812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c430643-97a2-41cf-9e43-325ce56a67bb None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:30.241195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c430643-97a2-41cf-9e43-325ce56a67bb None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:30.241477 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c430643-97a2-41cf-9e43-325ce56a67bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:30.259992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-8c430643-97a2-41cf-9e43-325ce56a67bb None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:30.261178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8c430643-97a2-41cf-9e43-325ce56a67bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:30.269609 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1244/2497] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:30.273617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6193a-d501-404d-bf78-3585c7692a2b None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:30.283619 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6193a-d501-404d-bf78-3585c7692a2b None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:30.288169 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3162310-e057-45a5-b14c-eca732e9d0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:30.288815 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3162310-e057-45a5-b14c-eca732e9d0f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:30.289098 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3162310-e057-45a5-b14c-eca732e9d0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:30.289372 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3162310-e057-45a5-b14c-eca732e9d0f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:30.290165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3162310-e057-45a5-b14c-eca732e9d0f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:22:30.290475 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3162310-e057-45a5-b14c-eca732e9d0f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:30.292070 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3162310-e057-45a5-b14c-eca732e9d0f6 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:30.303792 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1254/2498] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:22:30.308023 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c430643-97a2-41cf-9e43-325ce56a67bb None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:30.317776 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c430643-97a2-41cf-9e43-325ce56a67bb None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:30.322785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0993810-7e87-4f38-838e-eab0091a6a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:30.323857 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0993810-7e87-4f38-838e-eab0091a6a73 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:30.324338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0993810-7e87-4f38-838e-eab0091a6a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:30.324629 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0993810-7e87-4f38-838e-eab0091a6a73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:30.326015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0993810-7e87-4f38-838e-eab0091a6a73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:30.328193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0993810-7e87-4f38-838e-eab0091a6a73 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:30.389059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1245/2499] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:22:30 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:22:30.393171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3162310-e057-45a5-b14c-eca732e9d0f6 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:30.403679 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3162310-e057-45a5-b14c-eca732e9d0f6 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:30.464651 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d564d-0a18-42c7-b13e-45d786ba6d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:30.465319 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d564d-0a18-42c7-b13e-45d786ba6d11 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:30.465609 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d564d-0a18-42c7-b13e-45d786ba6d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:30.465903 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d564d-0a18-42c7-b13e-45d786ba6d11 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:30.466634 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d564d-0a18-42c7-b13e-45d786ba6d11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:30.468544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d564d-0a18-42c7-b13e-45d786ba6d11 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:30.567685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1255/2500] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:22:30 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:22:30.571739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0993810-7e87-4f38-838e-eab0091a6a73 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:30.583101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0993810-7e87-4f38-838e-eab0091a6a73 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:30.587811 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbcb0b-f4b2-4538-a684-4a730df31da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:30.593503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbcb0b-f4b2-4538-a684-4a730df31da0 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:30.593503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbcb0b-f4b2-4538-a684-4a730df31da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:30.593503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbcb0b-f4b2-4538-a684-4a730df31da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:30.593503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbcb0b-f4b2-4538-a684-4a730df31da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:30.593503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbcb0b-f4b2-4538-a684-4a730df31da0 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:30.606178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1246/2501] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:30.607702 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d564d-0a18-42c7-b13e-45d786ba6d11 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:30.617940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d564d-0a18-42c7-b13e-45d786ba6d11 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:30.623360 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae804063-c5d1-4200-94c4-6a95efe47cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:30.624023 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae804063-c5d1-4200-94c4-6a95efe47cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:30.624309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae804063-c5d1-4200-94c4-6a95efe47cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:30.624591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae804063-c5d1-4200-94c4-6a95efe47cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:30.625360 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae804063-c5d1-4200-94c4-6a95efe47cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:30.626999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae804063-c5d1-4200-94c4-6a95efe47cd4 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:30.637590 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1256/2502] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:30.642254 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbcb0b-f4b2-4538-a684-4a730df31da0 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:30.652833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbcb0b-f4b2-4538-a684-4a730df31da0 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:30.657708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b14202-96e3-4ed5-a05d-1b4b9f90e339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:30.658655 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b14202-96e3-4ed5-a05d-1b4b9f90e339 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:30.658926 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b14202-96e3-4ed5-a05d-1b4b9f90e339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:30.659201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b14202-96e3-4ed5-a05d-1b4b9f90e339 admin admin] PATH_INFO: `/v3/projects/117184717009478caf9f37865408bb51/users/d1c60aff10514aea98d8a9a9217e91f6/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:30.676002 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b14202-96e3-4ed5-a05d-1b4b9f90e339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=117184717009478caf9f37865408bb51, user_id=d1c60aff10514aea98d8a9a9217e91f6, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:30.678890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b14202-96e3-4ed5-a05d-1b4b9f90e339 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:30.780045 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1247/2503] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:22:30 2022] PUT /identity/v3/projects/117184717009478caf9f37865408bb51/users/d1c60aff10514aea98d8a9a9217e91f6/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:22:30.790480 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad57f34-a683-44c5-8ace-ee7e2085c696 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:30.790807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad57f34-a683-44c5-8ace-ee7e2085c696 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:30.791089 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad57f34-a683-44c5-8ace-ee7e2085c696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:30.818481 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-8ad57f34-a683-44c5-8ace-ee7e2085c696 None None] MFA Rules not processed for user `d1c60aff10514aea98d8a9a9217e91f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:30.819661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-8ad57f34-a683-44c5-8ace-ee7e2085c696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:30.880988 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1257/2504] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:30.892079 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b14202-96e3-4ed5-a05d-1b4b9f90e339 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:30.893844 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-29b14202-96e3-4ed5-a05d-1b4b9f90e339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:30.949427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b14202-96e3-4ed5-a05d-1b4b9f90e339 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:30.953244 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d30f1-b203-4a5e-860e-d3c039299e69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:30.953883 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d30f1-b203-4a5e-860e-d3c039299e69 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:30.954197 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d30f1-b203-4a5e-860e-d3c039299e69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:30.954474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d30f1-b203-4a5e-860e-d3c039299e69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:30.969592 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d30f1-b203-4a5e-860e-d3c039299e69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:30.972348 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d30f1-b203-4a5e-860e-d3c039299e69 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:30.986974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1248/2505] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:31.005586 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad57f34-a683-44c5-8ace-ee7e2085c696 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:31.064708 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad57f34-a683-44c5-8ace-ee7e2085c696 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:31.069288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c419c-8ad1-4db0-9d59-1f7801d41253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:31.070166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c419c-8ad1-4db0-9d59-1f7801d41253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:31.070456 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c419c-8ad1-4db0-9d59-1f7801d41253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:31.070731 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c419c-8ad1-4db0-9d59-1f7801d41253 admin admin] PATH_INFO: `/v3/users/d1c60aff10514aea98d8a9a9217e91f6` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:31.071926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c419c-8ad1-4db0-9d59-1f7801d41253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c60aff10514aea98d8a9a9217e91f6)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:31.073838 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c419c-8ad1-4db0-9d59-1f7801d41253 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:31.238602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-fc6c419c-8ad1-4db0-9d59-1f7801d41253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c60aff10514aea98d8a9a9217e91f6'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:31.239247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-fc6c419c-8ad1-4db0-9d59-1f7801d41253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c60aff10514aea98d8a9a9217e91f6'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:31.239822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-fc6c419c-8ad1-4db0-9d59-1f7801d41253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c60aff10514aea98d8a9a9217e91f6'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:31.251867 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-fc6c419c-8ad1-4db0-9d59-1f7801d41253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c60aff10514aea98d8a9a9217e91f6'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:31.258885 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-fc6c419c-8ad1-4db0-9d59-1f7801d41253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c60aff10514aea98d8a9a9217e91f6'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:31.322044 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1258/2506] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:22:31 2022] DELETE /identity/v3/users/d1c60aff10514aea98d8a9a9217e91f6 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:31.327437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29544da-a6f3-4232-84af-c4a0f5bb0f73 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:31.327730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29544da-a6f3-4232-84af-c4a0f5bb0f73 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:31.328292 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29544da-a6f3-4232-84af-c4a0f5bb0f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:31.344908 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-f29544da-a6f3-4232-84af-c4a0f5bb0f73 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:31.346136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f29544da-a6f3-4232-84af-c4a0f5bb0f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:31.403733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1249/2507] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:31.413282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c419c-8ad1-4db0-9d59-1f7801d41253 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:31.414694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-fc6c419c-8ad1-4db0-9d59-1f7801d41253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:31.472315 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c419c-8ad1-4db0-9d59-1f7801d41253 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:31.476059 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e13f3-be5a-452d-a5bc-e0b80aefccca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:31.476707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e13f3-be5a-452d-a5bc-e0b80aefccca service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:31.476992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e13f3-be5a-452d-a5bc-e0b80aefccca service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:31.477274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e13f3-be5a-452d-a5bc-e0b80aefccca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:31.486206 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e13f3-be5a-452d-a5bc-e0b80aefccca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:31.487624 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e13f3-be5a-452d-a5bc-e0b80aefccca service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:31.496946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1259/2508] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:31.907772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29544da-a6f3-4232-84af-c4a0f5bb0f73 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:31.909809 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f29544da-a6f3-4232-84af-c4a0f5bb0f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:31.922995 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29544da-a6f3-4232-84af-c4a0f5bb0f73 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:31.935618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bd715-270b-4e68-88d0-e60c726b4dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:31.936512 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bd715-270b-4e68-88d0-e60c726b4dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:31.936799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bd715-270b-4e68-88d0-e60c726b4dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:31.937073 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bd715-270b-4e68-88d0-e60c726b4dd3 admin admin] PATH_INFO: `/v3/projects/117184717009478caf9f37865408bb51` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:31.938392 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bd715-270b-4e68-88d0-e60c726b4dd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=117184717009478caf9f37865408bb51)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:31.940356 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bd715-270b-4e68-88d0-e60c726b4dd3 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:32.054566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-8c2bd715-270b-4e68-88d0-e60c726b4dd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '117184717009478caf9f37865408bb51', 'request_id': 'req-8c2bd715-270b-4e68-88d0-e60c726b4dd3'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:32.116600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-8c2bd715-270b-4e68-88d0-e60c726b4dd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '117184717009478caf9f37865408bb51', 'request_id': 'req-8c2bd715-270b-4e68-88d0-e60c726b4dd3'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:32.123041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-8c2bd715-270b-4e68-88d0-e60c726b4dd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '117184717009478caf9f37865408bb51', 'request_id': 'req-8c2bd715-270b-4e68-88d0-e60c726b4dd3'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:32.133806 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-8c2bd715-270b-4e68-88d0-e60c726b4dd3 admin admin] The token cache is being invalidate because project 117184717009478caf9f37865408bb51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:22:32.134104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-8c2bd715-270b-4e68-88d0-e60c726b4dd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:32.134751 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-8c2bd715-270b-4e68-88d0-e60c726b4dd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:32.136289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1250/2509] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:22:31 2022] DELETE /identity/v3/projects/117184717009478caf9f37865408bb51 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:22:32.141332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912b69d-d3d7-4201-9f90-1ada3573c294 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:32.141701 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912b69d-d3d7-4201-9f90-1ada3573c294 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:32.141988 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912b69d-d3d7-4201-9f90-1ada3573c294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:32.159344 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-a912b69d-d3d7-4201-9f90-1ada3573c294 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:32.160591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a912b69d-d3d7-4201-9f90-1ada3573c294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:32.219501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1260/2510] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:22:32.228220 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f262b-7047-43a4-a9f2-dbd37973a7f1 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:32.228515 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f262b-7047-43a4-a9f2-dbd37973a7f1 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:32.228796 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f262b-7047-43a4-a9f2-dbd37973a7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:32.248081 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-ba3f262b-7047-43a4-a9f2-dbd37973a7f1 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:32.249308 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-ba3f262b-7047-43a4-a9f2-dbd37973a7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:32.257335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1251/2511] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:32.261372 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912b69d-d3d7-4201-9f90-1ada3573c294 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:32.270861 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912b69d-d3d7-4201-9f90-1ada3573c294 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:32.275325 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec5245-b79b-45b9-9c01-d3da8e9f5961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:32.275960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec5245-b79b-45b9-9c01-d3da8e9f5961 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:32.276241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec5245-b79b-45b9-9c01-d3da8e9f5961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:32.276509 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec5245-b79b-45b9-9c01-d3da8e9f5961 admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:32.277230 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec5245-b79b-45b9-9c01-d3da8e9f5961 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:22:32.277531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec5245-b79b-45b9-9c01-d3da8e9f5961 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:32.279285 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec5245-b79b-45b9-9c01-d3da8e9f5961 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:32.290282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1261/2512] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:22:32.294801 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f262b-7047-43a4-a9f2-dbd37973a7f1 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:32.303798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f262b-7047-43a4-a9f2-dbd37973a7f1 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:32.308486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912bdd4-a090-40aa-89e6-6f124727fc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:32.309069 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912bdd4-a090-40aa-89e6-6f124727fc10 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:32.309351 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912bdd4-a090-40aa-89e6-6f124727fc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:32.310205 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912bdd4-a090-40aa-89e6-6f124727fc10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:32.310916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912bdd4-a090-40aa-89e6-6f124727fc10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:32.312792 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912bdd4-a090-40aa-89e6-6f124727fc10 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:32.421380 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1252/2513] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:22:32 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:22:32.425276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec5245-b79b-45b9-9c01-d3da8e9f5961 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:32.436144 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec5245-b79b-45b9-9c01-d3da8e9f5961 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:32.491509 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e487b6-fb36-425b-bf02-5307d69bce92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:32.492174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e487b6-fb36-425b-bf02-5307d69bce92 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:32.492454 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e487b6-fb36-425b-bf02-5307d69bce92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:32.492728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e487b6-fb36-425b-bf02-5307d69bce92 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:32.493455 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e487b6-fb36-425b-bf02-5307d69bce92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:32.495375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e487b6-fb36-425b-bf02-5307d69bce92 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:32.585681 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1262/2514] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:22:32 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:22:32.589445 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912bdd4-a090-40aa-89e6-6f124727fc10 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:32.600385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912bdd4-a090-40aa-89e6-6f124727fc10 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:32.605219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21709364-d987-4a42-8ca7-a5da3d013f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:32.605970 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21709364-d987-4a42-8ca7-a5da3d013f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:32.606277 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21709364-d987-4a42-8ca7-a5da3d013f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:32.606607 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21709364-d987-4a42-8ca7-a5da3d013f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:32.607338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21709364-d987-4a42-8ca7-a5da3d013f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:32.608952 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21709364-d987-4a42-8ca7-a5da3d013f57 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:32.619274 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1253/2515] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:22:32.623767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e487b6-fb36-425b-bf02-5307d69bce92 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:32.633861 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e487b6-fb36-425b-bf02-5307d69bce92 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:32.639355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ef7b4-8868-48b6-a53b-7fcfdf68f7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:32.640009 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ef7b4-8868-48b6-a53b-7fcfdf68f7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:32.640295 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ef7b4-8868-48b6-a53b-7fcfdf68f7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:32.640601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ef7b4-8868-48b6-a53b-7fcfdf68f7e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:32.641368 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ef7b4-8868-48b6-a53b-7fcfdf68f7e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:32.642975 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ef7b4-8868-48b6-a53b-7fcfdf68f7e9 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:32.652763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1263/2516] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:32.660580 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21709364-d987-4a42-8ca7-a5da3d013f57 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:32.677220 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21709364-d987-4a42-8ca7-a5da3d013f57 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:32.681940 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46d6ed-83a1-495a-bde1-61fb5228a676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:32.682845 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46d6ed-83a1-495a-bde1-61fb5228a676 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:32.683129 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46d6ed-83a1-495a-bde1-61fb5228a676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:32.683406 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46d6ed-83a1-495a-bde1-61fb5228a676 admin admin] PATH_INFO: `/v3/projects/847a71dec06441478a4635dcd37bca69/users/1926241f59aa48f9bed0868fd0988cde/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:32.697681 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46d6ed-83a1-495a-bde1-61fb5228a676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=847a71dec06441478a4635dcd37bca69, user_id=1926241f59aa48f9bed0868fd0988cde, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:32.700361 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46d6ed-83a1-495a-bde1-61fb5228a676 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:32.819987 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1254/2517] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:22:32 2022] PUT /identity/v3/projects/847a71dec06441478a4635dcd37bca69/users/1926241f59aa48f9bed0868fd0988cde/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:32.831789 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33fe39-439c-4325-97bc-e9bdf00da8d9 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:32.832042 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33fe39-439c-4325-97bc-e9bdf00da8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:32.832321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33fe39-439c-4325-97bc-e9bdf00da8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:32.858791 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-ac33fe39-439c-4325-97bc-e9bdf00da8d9 None None] MFA Rules not processed for user `1926241f59aa48f9bed0868fd0988cde`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:32.859988 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-ac33fe39-439c-4325-97bc-e9bdf00da8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:32.923018 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1264/2518] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:32.927687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46d6ed-83a1-495a-bde1-61fb5228a676 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:32.993529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46d6ed-83a1-495a-bde1-61fb5228a676 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:32.999034 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66469792-055a-449b-add8-e51b29e14d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:32.999703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66469792-055a-449b-add8-e51b29e14d34 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:33.000113 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66469792-055a-449b-add8-e51b29e14d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:33.000402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66469792-055a-449b-add8-e51b29e14d34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:33.001118 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66469792-055a-449b-add8-e51b29e14d34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:33.003144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66469792-055a-449b-add8-e51b29e14d34 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:33.103497 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1255/2519] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:22:32 2022] POST /identity/v3/projects => generated 375 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:22:33.107947 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33fe39-439c-4325-97bc-e9bdf00da8d9 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:33.119050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33fe39-439c-4325-97bc-e9bdf00da8d9 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:33.178138 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4978aa1-81fe-4410-89f3-b9026945edec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:33.179141 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4978aa1-81fe-4410-89f3-b9026945edec admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:33.179429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4978aa1-81fe-4410-89f3-b9026945edec admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:33.180008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4978aa1-81fe-4410-89f3-b9026945edec admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:33.180732 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4978aa1-81fe-4410-89f3-b9026945edec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:33.182674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4978aa1-81fe-4410-89f3-b9026945edec admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:33.277105 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1265/2520] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:22:33 2022] POST /identity/v3/users => generated 288 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:22:33.280966 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66469792-055a-449b-add8-e51b29e14d34 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:33.291147 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66469792-055a-449b-add8-e51b29e14d34 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:33.295783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aaee18-6e2b-4d68-b195-8a36bc64a766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:33.296437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aaee18-6e2b-4d68-b195-8a36bc64a766 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:33.296721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aaee18-6e2b-4d68-b195-8a36bc64a766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:33.296999 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aaee18-6e2b-4d68-b195-8a36bc64a766 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:33.297727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aaee18-6e2b-4d68-b195-8a36bc64a766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:33.299351 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aaee18-6e2b-4d68-b195-8a36bc64a766 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:33.310154 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1256/2521] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:33.313925 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4978aa1-81fe-4410-89f3-b9026945edec admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:33.323872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4978aa1-81fe-4410-89f3-b9026945edec admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:33.329243 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716adee3-2b31-497e-8a6c-0ee0a8bd8eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:33.329894 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716adee3-2b31-497e-8a6c-0ee0a8bd8eff admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:33.330217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716adee3-2b31-497e-8a6c-0ee0a8bd8eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:33.330499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716adee3-2b31-497e-8a6c-0ee0a8bd8eff admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:33.331230 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716adee3-2b31-497e-8a6c-0ee0a8bd8eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:33.332827 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716adee3-2b31-497e-8a6c-0ee0a8bd8eff admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:33.342459 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1266/2522] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:22:33.346314 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aaee18-6e2b-4d68-b195-8a36bc64a766 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:33.355967 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aaee18-6e2b-4d68-b195-8a36bc64a766 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:33.360502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92e02b-2416-4242-83b1-06cbe28a3f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:33.361362 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92e02b-2416-4242-83b1-06cbe28a3f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:33.361646 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92e02b-2416-4242-83b1-06cbe28a3f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:33.361924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92e02b-2416-4242-83b1-06cbe28a3f8f admin admin] PATH_INFO: `/v3/projects/388e17a2c57e44ddb1df5c435c381595/users/bacf21213d544775ab247f492a27196a/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:33.376401 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92e02b-2416-4242-83b1-06cbe28a3f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=388e17a2c57e44ddb1df5c435c381595, user_id=bacf21213d544775ab247f492a27196a, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:33.379106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92e02b-2416-4242-83b1-06cbe28a3f8f admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:33.448441 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1257/2523] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:22:33 2022] PUT /identity/v3/projects/388e17a2c57e44ddb1df5c435c381595/users/bacf21213d544775ab247f492a27196a/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:33.460594 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1826d78-396a-49d1-b5c0-cf8c172de8eb None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:33.460888 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1826d78-396a-49d1-b5c0-cf8c172de8eb None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:33.461165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1826d78-396a-49d1-b5c0-cf8c172de8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:33.486803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-d1826d78-396a-49d1-b5c0-cf8c172de8eb None None] MFA Rules not processed for user `bacf21213d544775ab247f492a27196a`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:33.488002 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-d1826d78-396a-49d1-b5c0-cf8c172de8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:33.550048 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1267/2524] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:33.554610 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92e02b-2416-4242-83b1-06cbe28a3f8f admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:33.615659 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92e02b-2416-4242-83b1-06cbe28a3f8f admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:33.620187 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913450dd-4ba9-47c6-bc37-8940da96fc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:33.620836 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913450dd-4ba9-47c6-bc37-8940da96fc5c admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:33.621116 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913450dd-4ba9-47c6-bc37-8940da96fc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:33.621387 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913450dd-4ba9-47c6-bc37-8940da96fc5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:33.622843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913450dd-4ba9-47c6-bc37-8940da96fc5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:33.624797 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913450dd-4ba9-47c6-bc37-8940da96fc5c admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:33.735130 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1258/2525] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:22:33 2022] POST /identity/v3/projects => generated 375 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:22:33.739175 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1826d78-396a-49d1-b5c0-cf8c172de8eb None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:33.749801 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1826d78-396a-49d1-b5c0-cf8c172de8eb None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:33.804100 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a1c92-725e-4095-ab1c-134b27581d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:33.804757 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a1c92-725e-4095-ab1c-134b27581d05 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:33.805041 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a1c92-725e-4095-ab1c-134b27581d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:33.805358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a1c92-725e-4095-ab1c-134b27581d05 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:33.806042 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a1c92-725e-4095-ab1c-134b27581d05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:33.808084 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a1c92-725e-4095-ab1c-134b27581d05 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:33.915432 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1268/2526] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:22:33 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:22:33.920007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913450dd-4ba9-47c6-bc37-8940da96fc5c admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:33.930196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913450dd-4ba9-47c6-bc37-8940da96fc5c admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:33.935526 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c6921-f40c-458c-8df3-498886072b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:33.936173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c6921-f40c-458c-8df3-498886072b5f admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:33.936452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c6921-f40c-458c-8df3-498886072b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:33.936746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c6921-f40c-458c-8df3-498886072b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:33.937462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704c6921-f40c-458c-8df3-498886072b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:33.939078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704c6921-f40c-458c-8df3-498886072b5f admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:33.949494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1259/2527] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:33.953654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a1c92-725e-4095-ab1c-134b27581d05 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:33.963551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a1c92-725e-4095-ab1c-134b27581d05 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:33.968972 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22c6ab-495c-45b7-9fed-808450104c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:33.969868 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22c6ab-495c-45b7-9fed-808450104c8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:33.970182 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22c6ab-495c-45b7-9fed-808450104c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:33.970458 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22c6ab-495c-45b7-9fed-808450104c8c admin admin] PATH_INFO: `/v3/projects/584c2eb0caec456982e79274ab0fa200/users/ef47116d717745509f6dd357db42f7e6/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:33.985286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e22c6ab-495c-45b7-9fed-808450104c8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=584c2eb0caec456982e79274ab0fa200, user_id=ef47116d717745509f6dd357db42f7e6, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:33.987952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e22c6ab-495c-45b7-9fed-808450104c8c admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:34.059353 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1269/2528] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:22:33 2022] PUT /identity/v3/projects/584c2eb0caec456982e79274ab0fa200/users/ef47116d717745509f6dd357db42f7e6/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:22:34.063813 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c6921-f40c-458c-8df3-498886072b5f admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:34.075881 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c6921-f40c-458c-8df3-498886072b5f admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:34.133776 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67eb08-bdee-4bc1-8639-1a962e01b7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:34.134487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67eb08-bdee-4bc1-8639-1a962e01b7c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:34.134778 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67eb08-bdee-4bc1-8639-1a962e01b7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:34.135080 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67eb08-bdee-4bc1-8639-1a962e01b7c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:34.135821 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67eb08-bdee-4bc1-8639-1a962e01b7c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:34.137421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67eb08-bdee-4bc1-8639-1a962e01b7c4 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:34.148332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1260/2529] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:34.152278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22c6ab-495c-45b7-9fed-808450104c8c admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:34.162498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22c6ab-495c-45b7-9fed-808450104c8c admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:34.167290 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80824fac-a2ea-4dc7-99d9-763187f9f66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:34.168112 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80824fac-a2ea-4dc7-99d9-763187f9f66c admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:34.168395 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80824fac-a2ea-4dc7-99d9-763187f9f66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:34.168674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80824fac-a2ea-4dc7-99d9-763187f9f66c admin admin] PATH_INFO: `/v3/projects/584c2eb0caec456982e79274ab0fa200/users/ef47116d717745509f6dd357db42f7e6/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:34.170725 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80824fac-a2ea-4dc7-99d9-763187f9f66c admin admin] RBAC: Authorizing `identity:create_grant(project_id=584c2eb0caec456982e79274ab0fa200, user_id=ef47116d717745509f6dd357db42f7e6, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:34.173353 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80824fac-a2ea-4dc7-99d9-763187f9f66c admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:34.184025 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1270/2530] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:22:34 2022] PUT /identity/v3/projects/584c2eb0caec456982e79274ab0fa200/users/ef47116d717745509f6dd357db42f7e6/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:22:34.195850 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3edb7-804e-46be-8f11-674a8a94bb0b None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:34.196145 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3edb7-804e-46be-8f11-674a8a94bb0b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:34.196427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3edb7-804e-46be-8f11-674a8a94bb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:34.223867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-78b3edb7-804e-46be-8f11-674a8a94bb0b None None] MFA Rules not processed for user `ef47116d717745509f6dd357db42f7e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:34.225085 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-78b3edb7-804e-46be-8f11-674a8a94bb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:34.297455 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1261/2531] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:34.308868 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80824fac-a2ea-4dc7-99d9-763187f9f66c admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:34.311177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-80824fac-a2ea-4dc7-99d9-763187f9f66c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:34.367842 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80824fac-a2ea-4dc7-99d9-763187f9f66c admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:34.374834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e86c19f-46cd-49b5-ac6c-637cee460c89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:34.374834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86c19f-46cd-49b5-ac6c-637cee460c89 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:34.374834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86c19f-46cd-49b5-ac6c-637cee460c89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:34.374834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86c19f-46cd-49b5-ac6c-637cee460c89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:34.443058 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e86c19f-46cd-49b5-ac6c-637cee460c89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:34.444491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e86c19f-46cd-49b5-ac6c-637cee460c89 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:34.453700 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1271/2532] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:36.409675 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3edb7-804e-46be-8f11-674a8a94bb0b None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:36.411075 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-78b3edb7-804e-46be-8f11-674a8a94bb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:36.422956 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3edb7-804e-46be-8f11-674a8a94bb0b None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:36.427167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32847b-ac5b-4caf-bbe6-5b7bbfd82e33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:36.427309 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32847b-ac5b-4caf-bbe6-5b7bbfd82e33 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:36.427513 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32847b-ac5b-4caf-bbe6-5b7bbfd82e33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:36.427770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32847b-ac5b-4caf-bbe6-5b7bbfd82e33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:36.490298 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32847b-ac5b-4caf-bbe6-5b7bbfd82e33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:36.491751 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32847b-ac5b-4caf-bbe6-5b7bbfd82e33 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:36.501672 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1262/2533] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:37.990969 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e86c19f-46cd-49b5-ac6c-637cee460c89 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:38.000243 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e86c19f-46cd-49b5-ac6c-637cee460c89 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:38.003898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f27764-4e02-41cd-811c-53133105be71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:38.004538 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f27764-4e02-41cd-811c-53133105be71 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:38.004821 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f27764-4e02-41cd-811c-53133105be71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:38.005093 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f27764-4e02-41cd-811c-53133105be71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:38.014719 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f27764-4e02-41cd-811c-53133105be71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:38.016134 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f27764-4e02-41cd-811c-53133105be71 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:38.025363 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1272/2534] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:37 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:42.650972 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32847b-ac5b-4caf-bbe6-5b7bbfd82e33 service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:42.714919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32847b-ac5b-4caf-bbe6-5b7bbfd82e33 service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:42.720137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9190f0-9137-4f90-919f-3a1994572b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:42.721102 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9190f0-9137-4f90-919f-3a1994572b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:42.721378 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9190f0-9137-4f90-919f-3a1994572b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:42.721648 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9190f0-9137-4f90-919f-3a1994572b23 admin admin] PATH_INFO: `/v3/users/1926241f59aa48f9bed0868fd0988cde` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:42.722842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9190f0-9137-4f90-919f-3a1994572b23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1926241f59aa48f9bed0868fd0988cde)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:42.724735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9190f0-9137-4f90-919f-3a1994572b23 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:42.930437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-9a9190f0-9137-4f90-919f-3a1994572b23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1926241f59aa48f9bed0868fd0988cde'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:42.931084 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-9a9190f0-9137-4f90-919f-3a1994572b23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1926241f59aa48f9bed0868fd0988cde'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:42.942946 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-9a9190f0-9137-4f90-919f-3a1994572b23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1926241f59aa48f9bed0868fd0988cde'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:42.943596 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-9a9190f0-9137-4f90-919f-3a1994572b23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1926241f59aa48f9bed0868fd0988cde'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:42.950195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-9a9190f0-9137-4f90-919f-3a1994572b23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1926241f59aa48f9bed0868fd0988cde'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:43.025216 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1263/2535] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:22:42 2022] DELETE /identity/v3/users/1926241f59aa48f9bed0868fd0988cde => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:43.030275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082fd41-fbb6-4118-8600-73e389c5dbf4 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:43.030752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082fd41-fbb6-4118-8600-73e389c5dbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:43.030851 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082fd41-fbb6-4118-8600-73e389c5dbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:43.047931 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-a082fd41-fbb6-4118-8600-73e389c5dbf4 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:43.049121 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a082fd41-fbb6-4118-8600-73e389c5dbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:43.106723 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1273/2536] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:43.116366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9190f0-9137-4f90-919f-3a1994572b23 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:43.117836 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-9a9190f0-9137-4f90-919f-3a1994572b23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:43.175289 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9190f0-9137-4f90-919f-3a1994572b23 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:43.179131 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdbc32-e0cc-4878-9492-5d8ad85d03e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:43.179784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdbc32-e0cc-4878-9492-5d8ad85d03e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:43.180066 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdbc32-e0cc-4878-9492-5d8ad85d03e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:43.180336 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdbc32-e0cc-4878-9492-5d8ad85d03e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:43.192984 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdbc32-e0cc-4878-9492-5d8ad85d03e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:43.192984 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdbc32-e0cc-4878-9492-5d8ad85d03e1 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:43.201275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1264/2537] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:43.675772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082fd41-fbb6-4118-8600-73e389c5dbf4 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:43.678489 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a082fd41-fbb6-4118-8600-73e389c5dbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:43.707394 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082fd41-fbb6-4118-8600-73e389c5dbf4 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:43.713534 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd5874-328b-4cf0-855c-ca08dae31e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:43.714799 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd5874-328b-4cf0-855c-ca08dae31e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:43.715146 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd5874-328b-4cf0-855c-ca08dae31e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:43.715537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd5874-328b-4cf0-855c-ca08dae31e47 admin admin] PATH_INFO: `/v3/projects/847a71dec06441478a4635dcd37bca69` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:43.717776 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd5874-328b-4cf0-855c-ca08dae31e47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=847a71dec06441478a4635dcd37bca69)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:43.719792 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd5874-328b-4cf0-855c-ca08dae31e47 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:43.860334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-78bd5874-328b-4cf0-855c-ca08dae31e47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '847a71dec06441478a4635dcd37bca69', 'request_id': 'req-78bd5874-328b-4cf0-855c-ca08dae31e47'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:43.927240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-78bd5874-328b-4cf0-855c-ca08dae31e47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '847a71dec06441478a4635dcd37bca69', 'request_id': 'req-78bd5874-328b-4cf0-855c-ca08dae31e47'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:43.932850 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-78bd5874-328b-4cf0-855c-ca08dae31e47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '847a71dec06441478a4635dcd37bca69', 'request_id': 'req-78bd5874-328b-4cf0-855c-ca08dae31e47'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:43.943611 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-78bd5874-328b-4cf0-855c-ca08dae31e47 admin admin] The token cache is being invalidate because project 847a71dec06441478a4635dcd37bca69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:22:43.943896 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-78bd5874-328b-4cf0-855c-ca08dae31e47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:43.944522 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-78bd5874-328b-4cf0-855c-ca08dae31e47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:43.946072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1274/2538] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:22:43 2022] DELETE /identity/v3/projects/847a71dec06441478a4635dcd37bca69 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:43.950045 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdbc32-e0cc-4878-9492-5d8ad85d03e1 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:43.959718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdbc32-e0cc-4878-9492-5d8ad85d03e1 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:44.014465 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ec9b3-f010-4709-8584-b3994ba46018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:44.015320 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ec9b3-f010-4709-8584-b3994ba46018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:44.015600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ec9b3-f010-4709-8584-b3994ba46018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:44.015873 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ec9b3-f010-4709-8584-b3994ba46018 admin admin] PATH_INFO: `/v3/users/bacf21213d544775ab247f492a27196a` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:44.017039 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ec9b3-f010-4709-8584-b3994ba46018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bacf21213d544775ab247f492a27196a)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:44.018961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ec9b3-f010-4709-8584-b3994ba46018 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:44.225517 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-ae0ec9b3-f010-4709-8584-b3994ba46018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bacf21213d544775ab247f492a27196a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:44.226210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-ae0ec9b3-f010-4709-8584-b3994ba46018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bacf21213d544775ab247f492a27196a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:44.238799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-ae0ec9b3-f010-4709-8584-b3994ba46018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bacf21213d544775ab247f492a27196a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:44.239428 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-ae0ec9b3-f010-4709-8584-b3994ba46018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bacf21213d544775ab247f492a27196a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:44.246070 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-ae0ec9b3-f010-4709-8584-b3994ba46018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bacf21213d544775ab247f492a27196a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:44.323080 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1265/2539] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:22:43 2022] DELETE /identity/v3/users/bacf21213d544775ab247f492a27196a => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:44.738571 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd5874-328b-4cf0-855c-ca08dae31e47 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:44.740165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-78bd5874-328b-4cf0-855c-ca08dae31e47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:44.773242 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd5874-328b-4cf0-855c-ca08dae31e47 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:44.840661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfdcef-5120-4c8d-9429-655479b8660c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:44.841795 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfdcef-5120-4c8d-9429-655479b8660c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:44.842213 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfdcef-5120-4c8d-9429-655479b8660c admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:44.842583 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfdcef-5120-4c8d-9429-655479b8660c admin admin] PATH_INFO: `/v3/projects/388e17a2c57e44ddb1df5c435c381595` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:44.845474 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfdcef-5120-4c8d-9429-655479b8660c admin admin] RBAC: Authorizing `identity:delete_project(project_id=388e17a2c57e44ddb1df5c435c381595)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:44.847886 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfdcef-5120-4c8d-9429-655479b8660c admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:44.991903 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e1cfdcef-5120-4c8d-9429-655479b8660c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '388e17a2c57e44ddb1df5c435c381595', 'request_id': 'req-e1cfdcef-5120-4c8d-9429-655479b8660c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:44.999924 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9a20e-7c5b-46be-8057-15f61ecb31a1 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:45.000418 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9a20e-7c5b-46be-8057-15f61ecb31a1 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:45.000522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9a20e-7c5b-46be-8057-15f61ecb31a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:45.019036 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-89a9a20e-7c5b-46be-8057-15f61ecb31a1 None None] MFA Rules not processed for user `7ec2605fd1b84e0ea134b943ed33a605`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:45.020167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-89a9a20e-7c5b-46be-8057-15f61ecb31a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:45.060232 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e1cfdcef-5120-4c8d-9429-655479b8660c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '388e17a2c57e44ddb1df5c435c381595', 'request_id': 'req-e1cfdcef-5120-4c8d-9429-655479b8660c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:45.066852 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e1cfdcef-5120-4c8d-9429-655479b8660c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '388e17a2c57e44ddb1df5c435c381595', 'request_id': 'req-e1cfdcef-5120-4c8d-9429-655479b8660c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:45.074872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1266/2540] 10.222.0.65 () {58 vars in 1009 bytes} [Tue May 10 20:22:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:22:45.079212 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e1cfdcef-5120-4c8d-9429-655479b8660c admin admin] The token cache is being invalidate because project 388e17a2c57e44ddb1df5c435c381595 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:22:45.079505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e1cfdcef-5120-4c8d-9429-655479b8660c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:45.080137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e1cfdcef-5120-4c8d-9429-655479b8660c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:45.081780 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1275/2541] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:22:44 2022] DELETE /identity/v3/projects/388e17a2c57e44ddb1df5c435c381595 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:45.085549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9a20e-7c5b-46be-8057-15f61ecb31a1 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:45.087032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-89a9a20e-7c5b-46be-8057-15f61ecb31a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:45.151986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9a20e-7c5b-46be-8057-15f61ecb31a1 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:45.157208 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb81a8a-a88c-4cdd-b39f-7fef6f256f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:45.158065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb81a8a-a88c-4cdd-b39f-7fef6f256f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:45.158370 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb81a8a-a88c-4cdd-b39f-7fef6f256f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:45.158644 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb81a8a-a88c-4cdd-b39f-7fef6f256f07 admin admin] PATH_INFO: `/v3/users/ef47116d717745509f6dd357db42f7e6` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:45.159816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb81a8a-a88c-4cdd-b39f-7fef6f256f07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef47116d717745509f6dd357db42f7e6)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:45.161718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb81a8a-a88c-4cdd-b39f-7fef6f256f07 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:45.218351 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfdcef-5120-4c8d-9429-655479b8660c admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:45.218351 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-e1cfdcef-5120-4c8d-9429-655479b8660c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:45.282358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfdcef-5120-4c8d-9429-655479b8660c admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:45.287375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a54691-826c-4efe-983c-5001ae437e3c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d10b258a53484da22950e2b1db2e41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:45.288025 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a54691-826c-4efe-983c-5001ae437e3c service swift] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:45.288315 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a54691-826c-4efe-983c-5001ae437e3c service swift] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:45.288598 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a54691-826c-4efe-983c-5001ae437e3c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:45.290448 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b7a54691-826c-4efe-983c-5001ae437e3c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:45.300713 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a54691-826c-4efe-983c-5001ae437e3c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:45.302354 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a54691-826c-4efe-983c-5001ae437e3c service swift] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:45.310791 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1276/2542] 10.222.0.65 () {60 vars in 1384 bytes} [Tue May 10 20:22:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 10 20:22:45.370588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0cb81a8a-a88c-4cdd-b39f-7fef6f256f07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef47116d717745509f6dd357db42f7e6'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:45.371247 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0cb81a8a-a88c-4cdd-b39f-7fef6f256f07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef47116d717745509f6dd357db42f7e6'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:45.384478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0cb81a8a-a88c-4cdd-b39f-7fef6f256f07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef47116d717745509f6dd357db42f7e6'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:45.385511 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0cb81a8a-a88c-4cdd-b39f-7fef6f256f07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef47116d717745509f6dd357db42f7e6'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:45.393020 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0cb81a8a-a88c-4cdd-b39f-7fef6f256f07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef47116d717745509f6dd357db42f7e6'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:45.495736 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1267/2543] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:22:45 2022] DELETE /identity/v3/users/ef47116d717745509f6dd357db42f7e6 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:46.000929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a54691-826c-4efe-983c-5001ae437e3c service swift] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:46.000929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b7a54691-826c-4efe-983c-5001ae437e3c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:46.014224 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a54691-826c-4efe-983c-5001ae437e3c service swift] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:46.119474 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd54dc-5685-4f37-8b9c-f306d131fce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:46.120507 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd54dc-5685-4f37-8b9c-f306d131fce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:46.120882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd54dc-5685-4f37-8b9c-f306d131fce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:46.121253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd54dc-5685-4f37-8b9c-f306d131fce2 admin admin] PATH_INFO: `/v3/projects/584c2eb0caec456982e79274ab0fa200` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:46.123258 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd54dc-5685-4f37-8b9c-f306d131fce2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=584c2eb0caec456982e79274ab0fa200)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:46.125446 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd54dc-5685-4f37-8b9c-f306d131fce2 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:46.293796 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-73cd54dc-5685-4f37-8b9c-f306d131fce2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '584c2eb0caec456982e79274ab0fa200', 'request_id': 'req-73cd54dc-5685-4f37-8b9c-f306d131fce2'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:46.369557 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-73cd54dc-5685-4f37-8b9c-f306d131fce2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '584c2eb0caec456982e79274ab0fa200', 'request_id': 'req-73cd54dc-5685-4f37-8b9c-f306d131fce2'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:46.377491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-73cd54dc-5685-4f37-8b9c-f306d131fce2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '584c2eb0caec456982e79274ab0fa200', 'request_id': 'req-73cd54dc-5685-4f37-8b9c-f306d131fce2'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:46.391796 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-73cd54dc-5685-4f37-8b9c-f306d131fce2 admin admin] The token cache is being invalidate because project 584c2eb0caec456982e79274ab0fa200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:22:46.393277 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-73cd54dc-5685-4f37-8b9c-f306d131fce2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:46.394194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-73cd54dc-5685-4f37-8b9c-f306d131fce2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:46.396564 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1277/2544] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:22:45 2022] DELETE /identity/v3/projects/584c2eb0caec456982e79274ab0fa200 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:46.401175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31739de-ee0c-4ff4-8c8b-35829bbe8feb None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:46.406444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31739de-ee0c-4ff4-8c8b-35829bbe8feb None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:46.406444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31739de-ee0c-4ff4-8c8b-35829bbe8feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:46.429075 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-d31739de-ee0c-4ff4-8c8b-35829bbe8feb None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:46.431150 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-d31739de-ee0c-4ff4-8c8b-35829bbe8feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:46.475492 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd54dc-5685-4f37-8b9c-f306d131fce2 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:46.477188 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-73cd54dc-5685-4f37-8b9c-f306d131fce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:46.508330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1268/2545] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:46.516120 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8b99d-2733-4fe0-a22b-6a2f4907c9fa None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:46.516613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8b99d-2733-4fe0-a22b-6a2f4907c9fa None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:46.517100 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8b99d-2733-4fe0-a22b-6a2f4907c9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:46.539158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-12f8b99d-2733-4fe0-a22b-6a2f4907c9fa None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:46.540791 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-12f8b99d-2733-4fe0-a22b-6a2f4907c9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:46.556248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1269/2546] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:46.560857 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8b99d-2733-4fe0-a22b-6a2f4907c9fa None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:46.591806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd54dc-5685-4f37-8b9c-f306d131fce2 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:46.605897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117926b6-c939-42ea-b9c1-af5c134fa610 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:46.605897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117926b6-c939-42ea-b9c1-af5c134fa610 service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:46.605897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117926b6-c939-42ea-b9c1-af5c134fa610 service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:46.605897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117926b6-c939-42ea-b9c1-af5c134fa610 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:46.610027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8b99d-2733-4fe0-a22b-6a2f4907c9fa None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:46.613170 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-117926b6-c939-42ea-b9c1-af5c134fa610 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:46.619695 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42565ee9-bea9-45d1-b3a9-0d1e21af21d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:46.619695 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42565ee9-bea9-45d1-b3a9-0d1e21af21d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:46.620585 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42565ee9-bea9-45d1-b3a9-0d1e21af21d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:46.620935 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42565ee9-bea9-45d1-b3a9-0d1e21af21d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:46.623344 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42565ee9-bea9-45d1-b3a9-0d1e21af21d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:22:46.623732 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42565ee9-bea9-45d1-b3a9-0d1e21af21d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:46.625983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42565ee9-bea9-45d1-b3a9-0d1e21af21d1 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:46.634067 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117926b6-c939-42ea-b9c1-af5c134fa610 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:46.635847 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117926b6-c939-42ea-b9c1-af5c134fa610 service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:46.650975 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1278/2547] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:22:46.663233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1270/2548] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:22:46.666173 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117926b6-c939-42ea-b9c1-af5c134fa610 service nova] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:46.682245 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117926b6-c939-42ea-b9c1-af5c134fa610 service nova] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:46.688719 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8879bcf-d989-402e-a920-4c11e04733a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:46.690374 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8879bcf-d989-402e-a920-4c11e04733a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:46.690804 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8879bcf-d989-402e-a920-4c11e04733a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:46.691195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8879bcf-d989-402e-a920-4c11e04733a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:46.692446 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8879bcf-d989-402e-a920-4c11e04733a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:46.694508 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8879bcf-d989-402e-a920-4c11e04733a5 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:46.814415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1279/2549] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:22:46 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:22:46.818632 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42565ee9-bea9-45d1-b3a9-0d1e21af21d1 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:46.830216 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42565ee9-bea9-45d1-b3a9-0d1e21af21d1 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:46.906890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f564f-ac5b-4a5c-a202-7b5071791562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:46.907681 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f564f-ac5b-4a5c-a202-7b5071791562 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:46.908028 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f564f-ac5b-4a5c-a202-7b5071791562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:46.908367 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f564f-ac5b-4a5c-a202-7b5071791562 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:46.909136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f564f-ac5b-4a5c-a202-7b5071791562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:46.911219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f564f-ac5b-4a5c-a202-7b5071791562 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:46.996476 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1271/2550] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:22:46 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:22:47.001223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8879bcf-d989-402e-a920-4c11e04733a5 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:47.011865 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8879bcf-d989-402e-a920-4c11e04733a5 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:47.020065 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f73844-2d1d-412f-a08a-5b04482d2012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:47.020844 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f73844-2d1d-412f-a08a-5b04482d2012 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:47.021134 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f73844-2d1d-412f-a08a-5b04482d2012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:47.021418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f73844-2d1d-412f-a08a-5b04482d2012 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:47.022197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f73844-2d1d-412f-a08a-5b04482d2012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:47.023873 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f73844-2d1d-412f-a08a-5b04482d2012 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:47.042220 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f564f-ac5b-4a5c-a202-7b5071791562 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:47.042761 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1280/2551] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:47.053694 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f564f-ac5b-4a5c-a202-7b5071791562 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:47.067055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fff7ab-65e9-4b4c-b494-f531ebcaedf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:47.067310 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fff7ab-65e9-4b4c-b494-f531ebcaedf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:47.067682 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fff7ab-65e9-4b4c-b494-f531ebcaedf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:47.068055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fff7ab-65e9-4b4c-b494-f531ebcaedf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:47.068868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fff7ab-65e9-4b4c-b494-f531ebcaedf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:47.070860 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fff7ab-65e9-4b4c-b494-f531ebcaedf6 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:47.092740 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1272/2552] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:47.093679 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f73844-2d1d-412f-a08a-5b04482d2012 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:47.112521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f73844-2d1d-412f-a08a-5b04482d2012 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:47.123437 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f551c8-e960-4e52-83c5-4f5f25ae96cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:47.124332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f551c8-e960-4e52-83c5-4f5f25ae96cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:47.124617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f551c8-e960-4e52-83c5-4f5f25ae96cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:47.124895 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f551c8-e960-4e52-83c5-4f5f25ae96cd admin admin] PATH_INFO: `/v3/projects/afa8a1f63379494c8777bc135ee46c02/users/5c73bf5ae67f42939d7a645d0f1e0800/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:47.160546 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f551c8-e960-4e52-83c5-4f5f25ae96cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=afa8a1f63379494c8777bc135ee46c02, user_id=5c73bf5ae67f42939d7a645d0f1e0800, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:47.163929 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f551c8-e960-4e52-83c5-4f5f25ae96cd admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:47.246102 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1281/2553] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:22:47 2022] PUT /identity/v3/projects/afa8a1f63379494c8777bc135ee46c02/users/5c73bf5ae67f42939d7a645d0f1e0800/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:22:47.257952 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07820e-9b58-4cd4-a9c8-f50bc354d278 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:47.259704 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07820e-9b58-4cd4-a9c8-f50bc354d278 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:47.259704 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07820e-9b58-4cd4-a9c8-f50bc354d278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:47.316022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-bf07820e-9b58-4cd4-a9c8-f50bc354d278 None None] MFA Rules not processed for user `5c73bf5ae67f42939d7a645d0f1e0800`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:47.316535 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-bf07820e-9b58-4cd4-a9c8-f50bc354d278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:47.414183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1273/2554] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:47.418392 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f551c8-e960-4e52-83c5-4f5f25ae96cd admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:47.504443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f551c8-e960-4e52-83c5-4f5f25ae96cd admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:47.513537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8bc58c-cad6-45ba-b379-aebe212ec7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:47.514249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8bc58c-cad6-45ba-b379-aebe212ec7ca admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:47.514534 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8bc58c-cad6-45ba-b379-aebe212ec7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:47.514807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8bc58c-cad6-45ba-b379-aebe212ec7ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:47.515546 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8bc58c-cad6-45ba-b379-aebe212ec7ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:47.517459 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8bc58c-cad6-45ba-b379-aebe212ec7ca admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:47.623132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1282/2555] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:22:47 2022] POST /identity/v3/projects => generated 383 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:22:47.627199 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07820e-9b58-4cd4-a9c8-f50bc354d278 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:47.638548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07820e-9b58-4cd4-a9c8-f50bc354d278 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:47.712118 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08587b07-2f82-4e21-85ce-66ccd83f3f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:47.712797 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08587b07-2f82-4e21-85ce-66ccd83f3f3d admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:47.713078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08587b07-2f82-4e21-85ce-66ccd83f3f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:47.713351 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08587b07-2f82-4e21-85ce-66ccd83f3f3d admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:47.714125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08587b07-2f82-4e21-85ce-66ccd83f3f3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:47.716111 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08587b07-2f82-4e21-85ce-66ccd83f3f3d admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:47.847378 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1274/2556] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:22:47 2022] POST /identity/v3/users => generated 292 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:22:47.848596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8bc58c-cad6-45ba-b379-aebe212ec7ca admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:47.867369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8bc58c-cad6-45ba-b379-aebe212ec7ca admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:47.875710 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e9329-2af0-403a-89cc-2fd1e8312442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:47.881072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e9329-2af0-403a-89cc-2fd1e8312442 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:47.881072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e9329-2af0-403a-89cc-2fd1e8312442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:47.881072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e9329-2af0-403a-89cc-2fd1e8312442 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:47.881072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e9329-2af0-403a-89cc-2fd1e8312442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:47.881072 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e9329-2af0-403a-89cc-2fd1e8312442 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:47.893741 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1283/2557] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:47.897757 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08587b07-2f82-4e21-85ce-66ccd83f3f3d admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:47.908766 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08587b07-2f82-4e21-85ce-66ccd83f3f3d admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:47.913623 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4529e-0b0e-4490-8869-34759c012611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:47.914395 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4529e-0b0e-4490-8869-34759c012611 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:47.914746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4529e-0b0e-4490-8869-34759c012611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:47.915107 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4529e-0b0e-4490-8869-34759c012611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:47.915909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4529e-0b0e-4490-8869-34759c012611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:47.917616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4529e-0b0e-4490-8869-34759c012611 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:47.930161 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1275/2558] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:47.933324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e9329-2af0-403a-89cc-2fd1e8312442 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:47.948748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e9329-2af0-403a-89cc-2fd1e8312442 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:47.953876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e8e40-ea7d-4564-acd6-144c579c8419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:47.954883 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e8e40-ea7d-4564-acd6-144c579c8419 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:47.956081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e8e40-ea7d-4564-acd6-144c579c8419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:47.956416 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e8e40-ea7d-4564-acd6-144c579c8419 admin admin] PATH_INFO: `/v3/projects/1572e5e2ed6840ea9032582713272f6b/users/d89a13ae255e4bf896f6283bfa07038a/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:47.985844 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e8e40-ea7d-4564-acd6-144c579c8419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1572e5e2ed6840ea9032582713272f6b, user_id=d89a13ae255e4bf896f6283bfa07038a, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:47.989098 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e8e40-ea7d-4564-acd6-144c579c8419 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:48.106400 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1284/2559] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:22:47 2022] PUT /identity/v3/projects/1572e5e2ed6840ea9032582713272f6b/users/d89a13ae255e4bf896f6283bfa07038a/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:22:48.118524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70672732-05f3-480a-8a1c-b140e7c0f0ec None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:48.119040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70672732-05f3-480a-8a1c-b140e7c0f0ec None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:48.119396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70672732-05f3-480a-8a1c-b140e7c0f0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:48.149612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-70672732-05f3-480a-8a1c-b140e7c0f0ec None None] MFA Rules not processed for user `d89a13ae255e4bf896f6283bfa07038a`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:48.150890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-70672732-05f3-480a-8a1c-b140e7c0f0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:48.226443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1276/2560] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:48.231281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e8e40-ea7d-4564-acd6-144c579c8419 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:48.297156 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e8e40-ea7d-4564-acd6-144c579c8419 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:48.314926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95d45a-1ba5-40f6-932f-1d814eafffac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:48.318191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95d45a-1ba5-40f6-932f-1d814eafffac admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:48.318191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95d45a-1ba5-40f6-932f-1d814eafffac admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:48.318191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95d45a-1ba5-40f6-932f-1d814eafffac admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:48.318191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95d45a-1ba5-40f6-932f-1d814eafffac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:48.319031 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95d45a-1ba5-40f6-932f-1d814eafffac admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:48.704725 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1285/2561] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:22:48 2022] POST /identity/v3/projects => generated 385 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:22:48.715292 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70672732-05f3-480a-8a1c-b140e7c0f0ec None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:48.726717 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70672732-05f3-480a-8a1c-b140e7c0f0ec None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:48.796359 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7e15c-2eba-47d1-9bbe-627e9a17c5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:48.797847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7e15c-2eba-47d1-9bbe-627e9a17c5b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:48.797847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7e15c-2eba-47d1-9bbe-627e9a17c5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:48.797847 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7e15c-2eba-47d1-9bbe-627e9a17c5b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:48.799883 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7e15c-2eba-47d1-9bbe-627e9a17c5b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:48.800373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7e15c-2eba-47d1-9bbe-627e9a17c5b4 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:48.920715 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1277/2562] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:22:48 2022] POST /identity/v3/users => generated 299 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:22:48.925619 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95d45a-1ba5-40f6-932f-1d814eafffac admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:48.949904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95d45a-1ba5-40f6-932f-1d814eafffac admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:48.962456 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe90b0a-c9ed-41a5-ba2e-6bacd67a98e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:48.963592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe90b0a-c9ed-41a5-ba2e-6bacd67a98e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:48.964114 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe90b0a-c9ed-41a5-ba2e-6bacd67a98e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:48.964562 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe90b0a-c9ed-41a5-ba2e-6bacd67a98e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:48.965476 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe90b0a-c9ed-41a5-ba2e-6bacd67a98e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:48.968433 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe90b0a-c9ed-41a5-ba2e-6bacd67a98e1 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:48.992772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1286/2563] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:48.993959 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7e15c-2eba-47d1-9bbe-627e9a17c5b4 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:49.005072 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7e15c-2eba-47d1-9bbe-627e9a17c5b4 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:49.011018 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b609f-8bf4-4594-a501-88134d99b805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:49.012140 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b609f-8bf4-4594-a501-88134d99b805 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:49.012426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b609f-8bf4-4594-a501-88134d99b805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:49.012700 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b609f-8bf4-4594-a501-88134d99b805 admin admin] PATH_INFO: `/v3/projects/468cb4e6560f43a299b2296b65cc2b63/users/4ed3935fa1394470a6ad12c222ae2eb1/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:49.033632 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b609f-8bf4-4594-a501-88134d99b805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=468cb4e6560f43a299b2296b65cc2b63, user_id=4ed3935fa1394470a6ad12c222ae2eb1, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:49.036856 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b609f-8bf4-4594-a501-88134d99b805 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:49.101855 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1278/2564] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:22:48 2022] PUT /identity/v3/projects/468cb4e6560f43a299b2296b65cc2b63/users/4ed3935fa1394470a6ad12c222ae2eb1/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:49.107645 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe90b0a-c9ed-41a5-ba2e-6bacd67a98e1 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:49.118806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe90b0a-c9ed-41a5-ba2e-6bacd67a98e1 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:49.191011 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72eb65-6e50-4dd8-911e-b1e0887505b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:49.191680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72eb65-6e50-4dd8-911e-b1e0887505b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:49.191961 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72eb65-6e50-4dd8-911e-b1e0887505b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:49.192240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72eb65-6e50-4dd8-911e-b1e0887505b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:49.192982 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72eb65-6e50-4dd8-911e-b1e0887505b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:49.194637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72eb65-6e50-4dd8-911e-b1e0887505b9 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:49.205811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1287/2565] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:22:49.210036 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b609f-8bf4-4594-a501-88134d99b805 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:49.224688 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b609f-8bf4-4594-a501-88134d99b805 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:49.231055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ac603-a5a2-448a-b824-d6122da957e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:49.233008 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ac603-a5a2-448a-b824-d6122da957e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:49.233459 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ac603-a5a2-448a-b824-d6122da957e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:49.233900 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ac603-a5a2-448a-b824-d6122da957e3 admin admin] PATH_INFO: `/v3/projects/468cb4e6560f43a299b2296b65cc2b63/users/4ed3935fa1394470a6ad12c222ae2eb1/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:49.236370 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ac603-a5a2-448a-b824-d6122da957e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=468cb4e6560f43a299b2296b65cc2b63, user_id=4ed3935fa1394470a6ad12c222ae2eb1, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:49.239065 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ac603-a5a2-448a-b824-d6122da957e3 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:49.253581 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1279/2566] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:22:49 2022] PUT /identity/v3/projects/468cb4e6560f43a299b2296b65cc2b63/users/4ed3935fa1394470a6ad12c222ae2eb1/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:49.265101 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42350efb-faee-46e2-a889-922bb00938c5 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:49.265391 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42350efb-faee-46e2-a889-922bb00938c5 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:49.265670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42350efb-faee-46e2-a889-922bb00938c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:49.301999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-42350efb-faee-46e2-a889-922bb00938c5 None None] MFA Rules not processed for user `4ed3935fa1394470a6ad12c222ae2eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:49.306444 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-42350efb-faee-46e2-a889-922bb00938c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:49.386133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1288/2567] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:49.397238 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ac603-a5a2-448a-b824-d6122da957e3 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:49.399032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1a8ac603-a5a2-448a-b824-d6122da957e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:49.474296 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ac603-a5a2-448a-b824-d6122da957e3 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:49.481084 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530bc1c-9178-4e9c-8fee-43f65ed28351 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:49.482714 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530bc1c-9178-4e9c-8fee-43f65ed28351 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:49.482996 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530bc1c-9178-4e9c-8fee-43f65ed28351 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:49.483273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530bc1c-9178-4e9c-8fee-43f65ed28351 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:49.553664 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530bc1c-9178-4e9c-8fee-43f65ed28351 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:49.557105 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530bc1c-9178-4e9c-8fee-43f65ed28351 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:49.569807 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1280/2568] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:51.656531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42350efb-faee-46e2-a889-922bb00938c5 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:51.666153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42350efb-faee-46e2-a889-922bb00938c5 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:51.670256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad30c9c-4f46-4dc4-9579-2415f5c5f30b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:51.670898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad30c9c-4f46-4dc4-9579-2415f5c5f30b service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:51.671179 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad30c9c-4f46-4dc4-9579-2415f5c5f30b service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:51.671455 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad30c9c-4f46-4dc4-9579-2415f5c5f30b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:51.730420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad30c9c-4f46-4dc4-9579-2415f5c5f30b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:51.731881 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad30c9c-4f46-4dc4-9579-2415f5c5f30b service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:51.744159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1289/2569] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:52.504712 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530bc1c-9178-4e9c-8fee-43f65ed28351 service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:52.506735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-2530bc1c-9178-4e9c-8fee-43f65ed28351 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:52.579010 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530bc1c-9178-4e9c-8fee-43f65ed28351 service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:52.583108 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3243995-0a98-4f73-a099-ae4dab610996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:52.584003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3243995-0a98-4f73-a099-ae4dab610996 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:52.584386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3243995-0a98-4f73-a099-ae4dab610996 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:52.587273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3243995-0a98-4f73-a099-ae4dab610996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:52.587273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-d3243995-0a98-4f73-a099-ae4dab610996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:52.601939 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3243995-0a98-4f73-a099-ae4dab610996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:52.603665 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3243995-0a98-4f73-a099-ae4dab610996 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:52.615291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1281/2570] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:53.810284 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad30c9c-4f46-4dc4-9579-2415f5c5f30b service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:53.821338 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad30c9c-4f46-4dc4-9579-2415f5c5f30b service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:53.828194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdf05e-3714-460c-a9d2-a58c3dec5d63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:53.834063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdf05e-3714-460c-a9d2-a58c3dec5d63 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:53.834063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdf05e-3714-460c-a9d2-a58c3dec5d63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:53.834063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdf05e-3714-460c-a9d2-a58c3dec5d63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:53.855188 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdf05e-3714-460c-a9d2-a58c3dec5d63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:53.856666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdf05e-3714-460c-a9d2-a58c3dec5d63 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:53.869030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1290/2571] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:58.815172 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3243995-0a98-4f73-a099-ae4dab610996 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:58.883491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3243995-0a98-4f73-a099-ae4dab610996 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:58.888277 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a390843-8184-4b60-a4af-6391af466d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:58.892567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a390843-8184-4b60-a4af-6391af466d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:58.892567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a390843-8184-4b60-a4af-6391af466d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:58.892567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a390843-8184-4b60-a4af-6391af466d6a admin admin] PATH_INFO: `/v3/users/5c73bf5ae67f42939d7a645d0f1e0800` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:58.892567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a390843-8184-4b60-a4af-6391af466d6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c73bf5ae67f42939d7a645d0f1e0800)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:58.892952 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a390843-8184-4b60-a4af-6391af466d6a admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:59.131515 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-2a390843-8184-4b60-a4af-6391af466d6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c73bf5ae67f42939d7a645d0f1e0800'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:59.132076 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-2a390843-8184-4b60-a4af-6391af466d6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c73bf5ae67f42939d7a645d0f1e0800'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:59.144409 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-2a390843-8184-4b60-a4af-6391af466d6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c73bf5ae67f42939d7a645d0f1e0800'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:59.145046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-2a390843-8184-4b60-a4af-6391af466d6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c73bf5ae67f42939d7a645d0f1e0800'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:59.151875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-2a390843-8184-4b60-a4af-6391af466d6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c73bf5ae67f42939d7a645d0f1e0800'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:22:59.214632 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1282/2572] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:22:58 2022] DELETE /identity/v3/users/5c73bf5ae67f42939d7a645d0f1e0800 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:22:59.219854 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e333f-9bf5-4205-bb67-5a62425735b9 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:59.220312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e333f-9bf5-4205-bb67-5a62425735b9 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:59.220409 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e333f-9bf5-4205-bb67-5a62425735b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:59.237974 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-536e333f-9bf5-4205-bb67-5a62425735b9 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:22:59.239193 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-536e333f-9bf5-4205-bb67-5a62425735b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:59.299815 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1291/2573] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:22:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:22:59.309316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a390843-8184-4b60-a4af-6391af466d6a admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:59.311648 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-2a390843-8184-4b60-a4af-6391af466d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:59.370849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a390843-8184-4b60-a4af-6391af466d6a admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:59.375082 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed2855-e825-4a56-b49f-eef0e893ff61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:59.375732 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed2855-e825-4a56-b49f-eef0e893ff61 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:59.376020 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed2855-e825-4a56-b49f-eef0e893ff61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:59.376297 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed2855-e825-4a56-b49f-eef0e893ff61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:59.386033 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed2855-e825-4a56-b49f-eef0e893ff61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:59.387502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed2855-e825-4a56-b49f-eef0e893ff61 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:59.397227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1283/2574] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:22:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:22:59.834812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e333f-9bf5-4205-bb67-5a62425735b9 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:22:59.836345 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-536e333f-9bf5-4205-bb67-5a62425735b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:22:59.850218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e333f-9bf5-4205-bb67-5a62425735b9 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:22:59.855517 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9495f2f-57ae-4de3-ab6f-4ad0b83d3942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:22:59.856386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9495f2f-57ae-4de3-ab6f-4ad0b83d3942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:22:59.856673 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9495f2f-57ae-4de3-ab6f-4ad0b83d3942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:22:59.857458 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9495f2f-57ae-4de3-ab6f-4ad0b83d3942 admin admin] PATH_INFO: `/v3/projects/afa8a1f63379494c8777bc135ee46c02` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:22:59.858710 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9495f2f-57ae-4de3-ab6f-4ad0b83d3942 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afa8a1f63379494c8777bc135ee46c02)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:22:59.860743 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9495f2f-57ae-4de3-ab6f-4ad0b83d3942 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:22:59.981186 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c9495f2f-57ae-4de3-ab6f-4ad0b83d3942 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afa8a1f63379494c8777bc135ee46c02', 'request_id': 'req-c9495f2f-57ae-4de3-ab6f-4ad0b83d3942'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:00.043501 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c9495f2f-57ae-4de3-ab6f-4ad0b83d3942 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afa8a1f63379494c8777bc135ee46c02', 'request_id': 'req-c9495f2f-57ae-4de3-ab6f-4ad0b83d3942'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:00.050648 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c9495f2f-57ae-4de3-ab6f-4ad0b83d3942 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afa8a1f63379494c8777bc135ee46c02', 'request_id': 'req-c9495f2f-57ae-4de3-ab6f-4ad0b83d3942'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:00.062733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c9495f2f-57ae-4de3-ab6f-4ad0b83d3942 admin admin] The token cache is being invalidate because project afa8a1f63379494c8777bc135ee46c02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:23:00.062990 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c9495f2f-57ae-4de3-ab6f-4ad0b83d3942 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:00.063806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-c9495f2f-57ae-4de3-ab6f-4ad0b83d3942 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:00.065469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1292/2575] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:22:59 2022] DELETE /identity/v3/projects/afa8a1f63379494c8777bc135ee46c02 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:00.069482 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed2855-e825-4a56-b49f-eef0e893ff61 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:00.081247 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed2855-e825-4a56-b49f-eef0e893ff61 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:00.159679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0694599-deec-45d7-b5a7-789da6d9f384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:00.160699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0694599-deec-45d7-b5a7-789da6d9f384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:00.165047 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0694599-deec-45d7-b5a7-789da6d9f384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:00.165047 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0694599-deec-45d7-b5a7-789da6d9f384 admin admin] PATH_INFO: `/v3/users/d89a13ae255e4bf896f6283bfa07038a` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:00.165047 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0694599-deec-45d7-b5a7-789da6d9f384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d89a13ae255e4bf896f6283bfa07038a)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:00.165634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0694599-deec-45d7-b5a7-789da6d9f384 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:00.381738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-c0694599-deec-45d7-b5a7-789da6d9f384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd89a13ae255e4bf896f6283bfa07038a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:00.382535 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-c0694599-deec-45d7-b5a7-789da6d9f384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd89a13ae255e4bf896f6283bfa07038a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:00.396507 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-c0694599-deec-45d7-b5a7-789da6d9f384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd89a13ae255e4bf896f6283bfa07038a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:00.397718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-c0694599-deec-45d7-b5a7-789da6d9f384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd89a13ae255e4bf896f6283bfa07038a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:00.405283 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-c0694599-deec-45d7-b5a7-789da6d9f384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd89a13ae255e4bf896f6283bfa07038a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:00.512870 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1284/2576] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:23:00 2022] DELETE /identity/v3/users/d89a13ae255e4bf896f6283bfa07038a => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:23:00.988060 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9495f2f-57ae-4de3-ab6f-4ad0b83d3942 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:00.990131 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-c9495f2f-57ae-4de3-ab6f-4ad0b83d3942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:01.005946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9495f2f-57ae-4de3-ab6f-4ad0b83d3942 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:01.066839 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c031d-533f-41c4-bb4a-6b0f562550bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:01.067706 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c031d-533f-41c4-bb4a-6b0f562550bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:01.067995 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c031d-533f-41c4-bb4a-6b0f562550bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:01.068274 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c031d-533f-41c4-bb4a-6b0f562550bd admin admin] PATH_INFO: `/v3/projects/1572e5e2ed6840ea9032582713272f6b` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:01.069458 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c031d-533f-41c4-bb4a-6b0f562550bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1572e5e2ed6840ea9032582713272f6b)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:01.071383 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c031d-533f-41c4-bb4a-6b0f562550bd admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:01.169518 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a28c031d-533f-41c4-bb4a-6b0f562550bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1572e5e2ed6840ea9032582713272f6b', 'request_id': 'req-a28c031d-533f-41c4-bb4a-6b0f562550bd'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:01.255642 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a28c031d-533f-41c4-bb4a-6b0f562550bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1572e5e2ed6840ea9032582713272f6b', 'request_id': 'req-a28c031d-533f-41c4-bb4a-6b0f562550bd'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:01.262124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a28c031d-533f-41c4-bb4a-6b0f562550bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1572e5e2ed6840ea9032582713272f6b', 'request_id': 'req-a28c031d-533f-41c4-bb4a-6b0f562550bd'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:01.273855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a28c031d-533f-41c4-bb4a-6b0f562550bd admin admin] The token cache is being invalidate because project 1572e5e2ed6840ea9032582713272f6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:23:01.274443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a28c031d-533f-41c4-bb4a-6b0f562550bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:01.275050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-a28c031d-533f-41c4-bb4a-6b0f562550bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:01.276736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1293/2577] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:23:00 2022] DELETE /identity/v3/projects/1572e5e2ed6840ea9032582713272f6b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:01.280836 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0694599-deec-45d7-b5a7-789da6d9f384 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:01.282334 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c0694599-deec-45d7-b5a7-789da6d9f384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:01.295541 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0694599-deec-45d7-b5a7-789da6d9f384 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:01.362186 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15109e-97f5-4790-9450-6f5151732e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:01.363102 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15109e-97f5-4790-9450-6f5151732e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:01.363386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15109e-97f5-4790-9450-6f5151732e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:01.365906 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15109e-97f5-4790-9450-6f5151732e93 admin admin] PATH_INFO: `/v3/users/4ed3935fa1394470a6ad12c222ae2eb1` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:01.365906 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e15109e-97f5-4790-9450-6f5151732e93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ed3935fa1394470a6ad12c222ae2eb1)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:01.366783 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e15109e-97f5-4790-9450-6f5151732e93 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:01.554386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4e15109e-97f5-4790-9450-6f5151732e93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ed3935fa1394470a6ad12c222ae2eb1'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:01.555040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4e15109e-97f5-4790-9450-6f5151732e93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ed3935fa1394470a6ad12c222ae2eb1'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:01.567632 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4e15109e-97f5-4790-9450-6f5151732e93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ed3935fa1394470a6ad12c222ae2eb1'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:01.568303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4e15109e-97f5-4790-9450-6f5151732e93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ed3935fa1394470a6ad12c222ae2eb1'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:01.575232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4e15109e-97f5-4790-9450-6f5151732e93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ed3935fa1394470a6ad12c222ae2eb1'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:01.632587 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1285/2578] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:23:01 2022] DELETE /identity/v3/users/4ed3935fa1394470a6ad12c222ae2eb1 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:23:02.197931 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c031d-533f-41c4-bb4a-6b0f562550bd admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:02.200106 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a28c031d-533f-41c4-bb4a-6b0f562550bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:02.214994 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c031d-533f-41c4-bb4a-6b0f562550bd admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:02.273623 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a354b5-5973-4bdb-b2b7-ac0f59ad729b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:02.274519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a354b5-5973-4bdb-b2b7-ac0f59ad729b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:02.274805 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a354b5-5973-4bdb-b2b7-ac0f59ad729b admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:02.275080 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a354b5-5973-4bdb-b2b7-ac0f59ad729b admin admin] PATH_INFO: `/v3/projects/468cb4e6560f43a299b2296b65cc2b63` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:02.276316 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a354b5-5973-4bdb-b2b7-ac0f59ad729b admin admin] RBAC: Authorizing `identity:delete_project(project_id=468cb4e6560f43a299b2296b65cc2b63)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:02.278381 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a354b5-5973-4bdb-b2b7-ac0f59ad729b admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:02.408826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-60a354b5-5973-4bdb-b2b7-ac0f59ad729b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '468cb4e6560f43a299b2296b65cc2b63', 'request_id': 'req-60a354b5-5973-4bdb-b2b7-ac0f59ad729b'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:02.488546 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-60a354b5-5973-4bdb-b2b7-ac0f59ad729b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '468cb4e6560f43a299b2296b65cc2b63', 'request_id': 'req-60a354b5-5973-4bdb-b2b7-ac0f59ad729b'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:02.495321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-60a354b5-5973-4bdb-b2b7-ac0f59ad729b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '468cb4e6560f43a299b2296b65cc2b63', 'request_id': 'req-60a354b5-5973-4bdb-b2b7-ac0f59ad729b'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:02.507334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-60a354b5-5973-4bdb-b2b7-ac0f59ad729b admin admin] The token cache is being invalidate because project 468cb4e6560f43a299b2296b65cc2b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:23:02.507627 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-60a354b5-5973-4bdb-b2b7-ac0f59ad729b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:02.508270 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-60a354b5-5973-4bdb-b2b7-ac0f59ad729b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:02.509870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1294/2579] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:23:02 2022] DELETE /identity/v3/projects/468cb4e6560f43a299b2296b65cc2b63 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:02.526222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d559eab-e5d1-4f81-924b-e5e09db87b40 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:02.526222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d559eab-e5d1-4f81-924b-e5e09db87b40 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:02.526222 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d559eab-e5d1-4f81-924b-e5e09db87b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:02.534328 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-1d559eab-e5d1-4f81-924b-e5e09db87b40 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:02.535574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1d559eab-e5d1-4f81-924b-e5e09db87b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:02.598355 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1286/2580] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:02.663516 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33c644-947b-4847-ad77-0299bbf0a526 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:02.663810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33c644-947b-4847-ad77-0299bbf0a526 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:02.664088 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33c644-947b-4847-ad77-0299bbf0a526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:02.687469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-7b33c644-947b-4847-ad77-0299bbf0a526 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:02.689040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7b33c644-947b-4847-ad77-0299bbf0a526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:02.698322 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1295/2581] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:02.707165 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d559eab-e5d1-4f81-924b-e5e09db87b40 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:02.718226 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d559eab-e5d1-4f81-924b-e5e09db87b40 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:02.724026 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbde47-af94-4c82-a236-7bbf2ffdfbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:02.724687 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbde47-af94-4c82-a236-7bbf2ffdfbbb admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:02.724970 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbde47-af94-4c82-a236-7bbf2ffdfbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:02.725244 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbde47-af94-4c82-a236-7bbf2ffdfbbb admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:02.725976 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbde47-af94-4c82-a236-7bbf2ffdfbbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:23:02.726308 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbde47-af94-4c82-a236-7bbf2ffdfbbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:02.727875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbde47-af94-4c82-a236-7bbf2ffdfbbb admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:02.740146 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1287/2582] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:23:02.744636 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b33c644-947b-4847-ad77-0299bbf0a526 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:02.754400 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b33c644-947b-4847-ad77-0299bbf0a526 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:02.759492 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc59edc-3283-4093-8e07-84671a08e62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:02.761507 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc59edc-3283-4093-8e07-84671a08e62d admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:02.761507 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc59edc-3283-4093-8e07-84671a08e62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:02.761507 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc59edc-3283-4093-8e07-84671a08e62d admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:02.761782 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc59edc-3283-4093-8e07-84671a08e62d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:02.764355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc59edc-3283-4093-8e07-84671a08e62d admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:02.896630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1296/2583] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:23:02 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:23:02.900539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbde47-af94-4c82-a236-7bbf2ffdfbbb admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:02.910550 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbde47-af94-4c82-a236-7bbf2ffdfbbb admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:02.973428 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac3b22-17de-4aad-8860-e7f18432d6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:02.974290 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac3b22-17de-4aad-8860-e7f18432d6fd admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:02.979143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac3b22-17de-4aad-8860-e7f18432d6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:02.979143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac3b22-17de-4aad-8860-e7f18432d6fd admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:02.979143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac3b22-17de-4aad-8860-e7f18432d6fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:02.979143 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac3b22-17de-4aad-8860-e7f18432d6fd admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:03.100555 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1288/2584] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:23:02 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:23:03.104384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc59edc-3283-4093-8e07-84671a08e62d admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:03.115138 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc59edc-3283-4093-8e07-84671a08e62d admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:03.120074 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6dceea-ef23-43c1-9b18-0d775f8af742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:03.120952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6dceea-ef23-43c1-9b18-0d775f8af742 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:03.121384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6dceea-ef23-43c1-9b18-0d775f8af742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:03.121819 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6dceea-ef23-43c1-9b18-0d775f8af742 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:03.122928 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6dceea-ef23-43c1-9b18-0d775f8af742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:03.124822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6dceea-ef23-43c1-9b18-0d775f8af742 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:03.136006 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1297/2585] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:23:03.140561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac3b22-17de-4aad-8860-e7f18432d6fd admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:03.150525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac3b22-17de-4aad-8860-e7f18432d6fd admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:03.155808 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f51b43-cd54-480d-8ed9-13178d3987a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:03.156465 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f51b43-cd54-480d-8ed9-13178d3987a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:03.159799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f51b43-cd54-480d-8ed9-13178d3987a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:03.159799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f51b43-cd54-480d-8ed9-13178d3987a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:03.159799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f51b43-cd54-480d-8ed9-13178d3987a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:03.159799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f51b43-cd54-480d-8ed9-13178d3987a2 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:03.172682 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1289/2586] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:23:03.174100 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6dceea-ef23-43c1-9b18-0d775f8af742 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:03.184538 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6dceea-ef23-43c1-9b18-0d775f8af742 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:03.189250 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d665d80-b01b-4058-85e0-754c637ea5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:03.190150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d665d80-b01b-4058-85e0-754c637ea5f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:03.190438 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d665d80-b01b-4058-85e0-754c637ea5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:03.190803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d665d80-b01b-4058-85e0-754c637ea5f4 admin admin] PATH_INFO: `/v3/projects/498ce5260ee44f5d9b1ebc74fecedfbf/users/77e46909519247119a6bc011b6bcfc7c/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:03.206995 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d665d80-b01b-4058-85e0-754c637ea5f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=498ce5260ee44f5d9b1ebc74fecedfbf, user_id=77e46909519247119a6bc011b6bcfc7c, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:03.209658 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d665d80-b01b-4058-85e0-754c637ea5f4 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:03.287599 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1298/2587] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:23:03 2022] PUT /identity/v3/projects/498ce5260ee44f5d9b1ebc74fecedfbf/users/77e46909519247119a6bc011b6bcfc7c/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:03.292759 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2ca96-0902-4728-9580-9e0df40213f7 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:03.293055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2ca96-0902-4728-9580-9e0df40213f7 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:03.293345 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2ca96-0902-4728-9580-9e0df40213f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:03.311367 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-2bb2ca96-0902-4728-9580-9e0df40213f7 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:03.312562 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-2bb2ca96-0902-4728-9580-9e0df40213f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:03.369283 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1290/2588] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:03.379438 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d665d80-b01b-4058-85e0-754c637ea5f4 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:03.381317 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-9d665d80-b01b-4058-85e0-754c637ea5f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:03.439699 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d665d80-b01b-4058-85e0-754c637ea5f4 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:03.443752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424dc2c-8999-4b91-8462-565f92c6f831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:03.444648 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424dc2c-8999-4b91-8462-565f92c6f831 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:03.445164 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424dc2c-8999-4b91-8462-565f92c6f831 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:03.445546 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424dc2c-8999-4b91-8462-565f92c6f831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:03.455632 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2424dc2c-8999-4b91-8462-565f92c6f831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:03.457500 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2424dc2c-8999-4b91-8462-565f92c6f831 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:03.467150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1299/2589] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:23:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:23:04.195340 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e89eb-97aa-415a-ab16-19eea4351afe None None] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:04.196195 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e89eb-97aa-415a-ab16-19eea4351afe None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:04.196851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e89eb-97aa-415a-ab16-19eea4351afe None None] PATH_INFO: `/` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:04.199113 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1291/2590] 10.222.0.65 () {54 vars in 921 bytes} [Tue May 10 20:23:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:23:04.208582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1384be2-16ee-4779-8946-1602a13f941f None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:04.209970 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1384be2-16ee-4779-8946-1602a13f941f None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:04.210491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1384be2-16ee-4779-8946-1602a13f941f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:04.231870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-c1384be2-16ee-4779-8946-1602a13f941f None None] MFA Rules not processed for user `ae922b2da4814d10bed1fa345700ac04`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:04.233018 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-c1384be2-16ee-4779-8946-1602a13f941f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:04.315044 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1300/2591] 10.222.0.65 () {58 vars in 1018 bytes} [Tue May 10 20:23:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:23:04.327410 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e89eb-97aa-415a-ab16-19eea4351afe None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:04.329560 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-6d5e89eb-97aa-415a-ab16-19eea4351afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:04.405993 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e89eb-97aa-415a-ab16-19eea4351afe None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:04.411704 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b253a-6cd1-4e03-a5a1-5d443df14ea1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67a252cde71a455fb1dc4bf9c47c0490', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:04.412608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b253a-6cd1-4e03-a5a1-5d443df14ea1 service glance] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:04.413101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b253a-6cd1-4e03-a5a1-5d443df14ea1 service glance] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:04.413619 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b253a-6cd1-4e03-a5a1-5d443df14ea1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:04.430619 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b253a-6cd1-4e03-a5a1-5d443df14ea1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:04.432832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b253a-6cd1-4e03-a5a1-5d443df14ea1 service glance] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:04.445994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1292/2592] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:23:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:23:04.645881 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991e2e4-c10d-4ddb-b636-ae48a6fbe186 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:04.646210 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991e2e4-c10d-4ddb-b636-ae48a6fbe186 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:04.646491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991e2e4-c10d-4ddb-b636-ae48a6fbe186 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:04.647886 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1301/2593] 10.222.0.65 () {54 vars in 921 bytes} [Tue May 10 20:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:23:04.653157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ce799-2550-4eae-9a6f-75e5441f05a3 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:04.653451 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ce799-2550-4eae-9a6f-75e5441f05a3 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:04.653851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ce799-2550-4eae-9a6f-75e5441f05a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:04.672437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-930ce799-2550-4eae-9a6f-75e5441f05a3 None None] MFA Rules not processed for user `a6aabd1571d04bcfae842626fef5ed6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:04.673640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-930ce799-2550-4eae-9a6f-75e5441f05a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:04.680583 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1293/2594] 10.222.0.65 () {58 vars in 1018 bytes} [Tue May 10 20:23:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:04.693515 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991e2e4-c10d-4ddb-b636-ae48a6fbe186 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:04.705328 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991e2e4-c10d-4ddb-b636-ae48a6fbe186 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:04.712829 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb0089-184e-41df-a388-b0c22ae43e43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:04.712829 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb0089-184e-41df-a388-b0c22ae43e43 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:04.712829 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb0089-184e-41df-a388-b0c22ae43e43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:04.712829 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb0089-184e-41df-a388-b0c22ae43e43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:04.717697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb0089-184e-41df-a388-b0c22ae43e43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:04.719133 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb0089-184e-41df-a388-b0c22ae43e43 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:04.726745 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1302/2595] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:23:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:23:04.896182 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf640984-d642-4c1c-ac13-ea6641428d8b None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:04.896875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf640984-d642-4c1c-ac13-ea6641428d8b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:04.897315 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf640984-d642-4c1c-ac13-ea6641428d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:04.916436 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a337e7-558c-483a-a26b-584f19b218f0 None None] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:04.917189 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a337e7-558c-483a-a26b-584f19b218f0 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:04.922344 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a337e7-558c-483a-a26b-584f19b218f0 None None] PATH_INFO: `/` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:04.922344 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1303/2596] 10.222.0.65 () {54 vars in 921 bytes} [Tue May 10 20:23:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 20:23:04.926853 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86e723-cc4a-44dd-bc84-da6c6e20ec6f None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:04.927387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86e723-cc4a-44dd-bc84-da6c6e20ec6f None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:04.927940 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86e723-cc4a-44dd-bc84-da6c6e20ec6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:04.933059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-cf640984-d642-4c1c-ac13-ea6641428d8b None None] MFA Rules not processed for user `77e46909519247119a6bc011b6bcfc7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:04.934669 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-cf640984-d642-4c1c-ac13-ea6641428d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:04.955119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-1e86e723-cc4a-44dd-bc84-da6c6e20ec6f None None] MFA Rules not processed for user `ae922b2da4814d10bed1fa345700ac04`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:04.957263 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-1e86e723-cc4a-44dd-bc84-da6c6e20ec6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:04.988428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1304/2597] 10.222.0.65 () {58 vars in 1018 bytes} [Tue May 10 20:23:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:23:05.004262 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86e723-cc4a-44dd-bc84-da6c6e20ec6f None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:05.016636 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86e723-cc4a-44dd-bc84-da6c6e20ec6f None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:05.020955 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c108b3-42a7-47d9-b2e6-9a383afb6ce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:05.021815 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c108b3-42a7-47d9-b2e6-9a383afb6ce2 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:05.022305 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c108b3-42a7-47d9-b2e6-9a383afb6ce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:05.022730 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c108b3-42a7-47d9-b2e6-9a383afb6ce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:05.036598 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c108b3-42a7-47d9-b2e6-9a383afb6ce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:05.038602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c108b3-42a7-47d9-b2e6-9a383afb6ce2 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:05.056535 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1305/2598] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:23:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:23:05.069734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1294/2599] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:05.087226 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c108b3-42a7-47d9-b2e6-9a383afb6ce2 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:05.088677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-15c108b3-42a7-47d9-b2e6-9a383afb6ce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:05.145997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c108b3-42a7-47d9-b2e6-9a383afb6ce2 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:05.151154 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e9721-2afd-4ab8-b486-945767c41b19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:05.151406 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e9721-2afd-4ab8-b486-945767c41b19 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:05.151629 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e9721-2afd-4ab8-b486-945767c41b19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:05.151905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e9721-2afd-4ab8-b486-945767c41b19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:05.162241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e9721-2afd-4ab8-b486-945767c41b19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:05.163963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e9721-2afd-4ab8-b486-945767c41b19 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:05.176289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1306/2600] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:23:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:23:06.905272 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925ab53-20da-4c09-a3ef-110cdbb6acdc None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:06.905773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925ab53-20da-4c09-a3ef-110cdbb6acdc None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:06.905870 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925ab53-20da-4c09-a3ef-110cdbb6acdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:06.923539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-3925ab53-20da-4c09-a3ef-110cdbb6acdc None None] MFA Rules not processed for user `77e46909519247119a6bc011b6bcfc7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:06.924740 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-3925ab53-20da-4c09-a3ef-110cdbb6acdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:06.931498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1295/2601] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:07.157019 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e9721-2afd-4ab8-b486-945767c41b19 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:07.166676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e9721-2afd-4ab8-b486-945767c41b19 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:07.170308 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f953ae-9858-4d78-a4dd-3fa816083bb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:07.171030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f953ae-9858-4d78-a4dd-3fa816083bb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:07.171313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f953ae-9858-4d78-a4dd-3fa816083bb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:07.171586 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f953ae-9858-4d78-a4dd-3fa816083bb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:07.173555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-33f953ae-9858-4d78-a4dd-3fa816083bb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:07.183426 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f953ae-9858-4d78-a4dd-3fa816083bb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:07.185450 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f953ae-9858-4d78-a4dd-3fa816083bb8 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:07.193521 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1307/2602] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:23:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:23:09.879866 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732111ae-785b-416c-a8ca-1c59d4ecad31 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:09.880400 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732111ae-785b-416c-a8ca-1c59d4ecad31 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:09.880741 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732111ae-785b-416c-a8ca-1c59d4ecad31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:09.922044 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-732111ae-785b-416c-a8ca-1c59d4ecad31 None None] MFA Rules not processed for user `7ec2605fd1b84e0ea134b943ed33a605`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:09.922044 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-732111ae-785b-416c-a8ca-1c59d4ecad31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:10.017464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1296/2603] 10.222.0.65 () {58 vars in 1009 bytes} [Tue May 10 20:23:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:10.043311 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f953ae-9858-4d78-a4dd-3fa816083bb8 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:10.046197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-33f953ae-9858-4d78-a4dd-3fa816083bb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:10.166074 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f953ae-9858-4d78-a4dd-3fa816083bb8 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:10.172408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4413038-8bf1-4ea3-8bcd-4bfa0768eb7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d10b258a53484da22950e2b1db2e41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:10.173086 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4413038-8bf1-4ea3-8bcd-4bfa0768eb7e service swift] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:10.173668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4413038-8bf1-4ea3-8bcd-4bfa0768eb7e service swift] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:10.174011 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4413038-8bf1-4ea3-8bcd-4bfa0768eb7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:10.184048 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4413038-8bf1-4ea3-8bcd-4bfa0768eb7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:10.186386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4413038-8bf1-4ea3-8bcd-4bfa0768eb7e service swift] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:10.197016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1308/2604] 10.222.0.65 () {60 vars in 1384 bytes} [Tue May 10 20:23:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 20:23:17.140835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732111ae-785b-416c-a8ca-1c59d4ecad31 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:17.140835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-732111ae-785b-416c-a8ca-1c59d4ecad31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:17.157412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732111ae-785b-416c-a8ca-1c59d4ecad31 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:17.165842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7c46d-5f52-4107-86a4-a18a22f89a52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:17.166814 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7c46d-5f52-4107-86a4-a18a22f89a52 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:17.167284 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7c46d-5f52-4107-86a4-a18a22f89a52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:17.167742 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7c46d-5f52-4107-86a4-a18a22f89a52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:17.171441 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-44e7c46d-5f52-4107-86a4-a18a22f89a52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:17.206044 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7c46d-5f52-4107-86a4-a18a22f89a52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:17.207553 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7c46d-5f52-4107-86a4-a18a22f89a52 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:17.253842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1297/2605] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:23:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:23:20.637650 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c84b8-98a5-48b5-86bc-b268b4b6db87 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:20.638243 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c84b8-98a5-48b5-86bc-b268b4b6db87 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:20.638988 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c84b8-98a5-48b5-86bc-b268b4b6db87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:20.672848 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-b98c84b8-98a5-48b5-86bc-b268b4b6db87 None None] MFA Rules not processed for user `46916c5b15254c529bc52ecb711fdd76`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:20.674990 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b98c84b8-98a5-48b5-86bc-b268b4b6db87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:20.788651 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1309/2606] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:21.967252 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7c46d-5f52-4107-86a4-a18a22f89a52 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:21.968876 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-44e7c46d-5f52-4107-86a4-a18a22f89a52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:21.980769 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7c46d-5f52-4107-86a4-a18a22f89a52 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:21.985988 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b773a-ecd6-46ed-8770-b8f5d2dc7b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:21.986888 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b773a-ecd6-46ed-8770-b8f5d2dc7b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:21.987171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b773a-ecd6-46ed-8770-b8f5d2dc7b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:21.987444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b773a-ecd6-46ed-8770-b8f5d2dc7b21 admin admin] PATH_INFO: `/v3/users/35e69662bc7648b8b3e7467cf56c7f5e` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:21.988646 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b773a-ecd6-46ed-8770-b8f5d2dc7b21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35e69662bc7648b8b3e7467cf56c7f5e)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:21.990585 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b773a-ecd6-46ed-8770-b8f5d2dc7b21 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:22.195918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-135b773a-ecd6-46ed-8770-b8f5d2dc7b21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35e69662bc7648b8b3e7467cf56c7f5e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:22.196773 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-135b773a-ecd6-46ed-8770-b8f5d2dc7b21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35e69662bc7648b8b3e7467cf56c7f5e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:22.211135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-135b773a-ecd6-46ed-8770-b8f5d2dc7b21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35e69662bc7648b8b3e7467cf56c7f5e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:22.212116 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-135b773a-ecd6-46ed-8770-b8f5d2dc7b21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35e69662bc7648b8b3e7467cf56c7f5e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:22.221693 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-135b773a-ecd6-46ed-8770-b8f5d2dc7b21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35e69662bc7648b8b3e7467cf56c7f5e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:22.314959 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1298/2607] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:23:21 2022] DELETE /identity/v3/users/35e69662bc7648b8b3e7467cf56c7f5e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:22.638294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c84b8-98a5-48b5-86bc-b268b4b6db87 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:22.649127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c84b8-98a5-48b5-86bc-b268b4b6db87 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:22.752841 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ca284-d38f-4699-876d-ccab4f5f8630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:22.753757 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ca284-d38f-4699-876d-ccab4f5f8630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:22.754886 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ca284-d38f-4699-876d-ccab4f5f8630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:22.755190 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ca284-d38f-4699-876d-ccab4f5f8630 admin admin] PATH_INFO: `/v3/projects/821dc2c3061041a3ab6ee939c306a7ea` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:22.757071 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ca284-d38f-4699-876d-ccab4f5f8630 admin admin] RBAC: Authorizing `identity:delete_project(project_id=821dc2c3061041a3ab6ee939c306a7ea)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:22.759508 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ca284-d38f-4699-876d-ccab4f5f8630 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:22.903487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5b1ca284-d38f-4699-876d-ccab4f5f8630 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '821dc2c3061041a3ab6ee939c306a7ea', 'request_id': 'req-5b1ca284-d38f-4699-876d-ccab4f5f8630'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:22.973371 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5b1ca284-d38f-4699-876d-ccab4f5f8630 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '821dc2c3061041a3ab6ee939c306a7ea', 'request_id': 'req-5b1ca284-d38f-4699-876d-ccab4f5f8630'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:22.981575 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5b1ca284-d38f-4699-876d-ccab4f5f8630 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '821dc2c3061041a3ab6ee939c306a7ea', 'request_id': 'req-5b1ca284-d38f-4699-876d-ccab4f5f8630'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:22.994440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5b1ca284-d38f-4699-876d-ccab4f5f8630 admin admin] The token cache is being invalidate because project 821dc2c3061041a3ab6ee939c306a7ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:23:22.994848 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5b1ca284-d38f-4699-876d-ccab4f5f8630 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:22.995596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-5b1ca284-d38f-4699-876d-ccab4f5f8630 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:23.000763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1310/2608] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:23:22 2022] DELETE /identity/v3/projects/821dc2c3061041a3ab6ee939c306a7ea => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:23.002355 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2caf1ec-6cc2-47ab-b27e-56ca00c87ea3 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:23.006359 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2caf1ec-6cc2-47ab-b27e-56ca00c87ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:23.006359 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2caf1ec-6cc2-47ab-b27e-56ca00c87ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:23.021835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-d2caf1ec-6cc2-47ab-b27e-56ca00c87ea3 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:23.023364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-d2caf1ec-6cc2-47ab-b27e-56ca00c87ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:23.091159 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1299/2609] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:23:23.099264 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc29223-27ee-4302-86cc-862b75d19f92 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:23.099564 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc29223-27ee-4302-86cc-862b75d19f92 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:23.099904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc29223-27ee-4302-86cc-862b75d19f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:23.123768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-efc29223-27ee-4302-86cc-862b75d19f92 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:23.125244 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-efc29223-27ee-4302-86cc-862b75d19f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:23.133432 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1311/2610] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:23.137862 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2caf1ec-6cc2-47ab-b27e-56ca00c87ea3 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:23.150438 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2caf1ec-6cc2-47ab-b27e-56ca00c87ea3 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:23.153703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531515b4-79e8-4972-95be-0b3be6ce86d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:23.155839 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531515b4-79e8-4972-95be-0b3be6ce86d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:23.156136 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531515b4-79e8-4972-95be-0b3be6ce86d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:23.156414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531515b4-79e8-4972-95be-0b3be6ce86d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:23.157198 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531515b4-79e8-4972-95be-0b3be6ce86d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:23:23.157511 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531515b4-79e8-4972-95be-0b3be6ce86d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:23.159649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531515b4-79e8-4972-95be-0b3be6ce86d2 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:23.180900 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1300/2611] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 20:23:23.187353 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc29223-27ee-4302-86cc-862b75d19f92 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:23.196995 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc29223-27ee-4302-86cc-862b75d19f92 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:23.203655 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c41f3-ea42-49d0-9d3f-88a1bc125dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:23.204673 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c41f3-ea42-49d0-9d3f-88a1bc125dae admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:23.205028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c41f3-ea42-49d0-9d3f-88a1bc125dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:23.205589 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c41f3-ea42-49d0-9d3f-88a1bc125dae admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:23.206972 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c41f3-ea42-49d0-9d3f-88a1bc125dae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:23.209267 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c41f3-ea42-49d0-9d3f-88a1bc125dae admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:23.287064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1312/2612] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:23:23 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:23:23.288207 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531515b4-79e8-4972-95be-0b3be6ce86d2 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:23.298486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531515b4-79e8-4972-95be-0b3be6ce86d2 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:23.354969 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276b6af-bf35-4be2-8fd5-b88cf14e873d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:23.355639 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276b6af-bf35-4be2-8fd5-b88cf14e873d admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:23.355921 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276b6af-bf35-4be2-8fd5-b88cf14e873d admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:23.356521 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276b6af-bf35-4be2-8fd5-b88cf14e873d admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:23.356934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4276b6af-bf35-4be2-8fd5-b88cf14e873d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:23.359410 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4276b6af-bf35-4be2-8fd5-b88cf14e873d admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:23.491856 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1301/2613] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:23:23 2022] POST /identity/v3/users => generated 285 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:23:23.495522 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c41f3-ea42-49d0-9d3f-88a1bc125dae admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:23.508074 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c41f3-ea42-49d0-9d3f-88a1bc125dae admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:23.516520 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23c74e-d8ee-4c56-9a7f-fd3597637ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:23.517318 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23c74e-d8ee-4c56-9a7f-fd3597637ccb admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:23.517807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23c74e-d8ee-4c56-9a7f-fd3597637ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:23.518837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23c74e-d8ee-4c56-9a7f-fd3597637ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:23.519677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23c74e-d8ee-4c56-9a7f-fd3597637ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:23.522040 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23c74e-d8ee-4c56-9a7f-fd3597637ccb admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:23.541427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1313/2614] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:23:23.542980 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276b6af-bf35-4be2-8fd5-b88cf14e873d admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:23.554737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276b6af-bf35-4be2-8fd5-b88cf14e873d admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:23.565236 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6a45b-62c3-4736-87e5-0fe2d45d3717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:23.566335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6a45b-62c3-4736-87e5-0fe2d45d3717 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:23.567226 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6a45b-62c3-4736-87e5-0fe2d45d3717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:23.567511 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6a45b-62c3-4736-87e5-0fe2d45d3717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:23.573688 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6a45b-62c3-4736-87e5-0fe2d45d3717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:23.573688 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6a45b-62c3-4736-87e5-0fe2d45d3717 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:23.600710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1302/2615] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:23:23.601336 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23c74e-d8ee-4c56-9a7f-fd3597637ccb admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:23.614957 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23c74e-d8ee-4c56-9a7f-fd3597637ccb admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:23.622545 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d0765-1aa4-43bf-be3a-791e9589500c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:23.627007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d0765-1aa4-43bf-be3a-791e9589500c admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:23.627007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d0765-1aa4-43bf-be3a-791e9589500c admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:23.627007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d0765-1aa4-43bf-be3a-791e9589500c admin admin] PATH_INFO: `/v3/projects/e9bc233a626643f0a6954adc2a215f6c/users/93b74480c62f4e179089f48b91bd7f28/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:23.647597 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d0765-1aa4-43bf-be3a-791e9589500c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9bc233a626643f0a6954adc2a215f6c, user_id=93b74480c62f4e179089f48b91bd7f28, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:23.651051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d0765-1aa4-43bf-be3a-791e9589500c admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:23.738488 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1314/2616] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:23:23 2022] PUT /identity/v3/projects/e9bc233a626643f0a6954adc2a215f6c/users/93b74480c62f4e179089f48b91bd7f28/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:23:23.750333 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65292ca-ba0c-440c-b13a-cc788d57cb73 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:23.750961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65292ca-ba0c-440c-b13a-cc788d57cb73 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:23.751470 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65292ca-ba0c-440c-b13a-cc788d57cb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:23.795507 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-f65292ca-ba0c-440c-b13a-cc788d57cb73 None None] MFA Rules not processed for user `93b74480c62f4e179089f48b91bd7f28`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:23.796823 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f65292ca-ba0c-440c-b13a-cc788d57cb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:23.870186 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1303/2617] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:23.880209 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d0765-1aa4-43bf-be3a-791e9589500c admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:23.882247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-759d0765-1aa4-43bf-be3a-791e9589500c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:23.956410 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d0765-1aa4-43bf-be3a-791e9589500c admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:23.962057 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b58380-ccb0-4224-add7-9828863af348 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:23.962927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b58380-ccb0-4224-add7-9828863af348 service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:23.963237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b58380-ccb0-4224-add7-9828863af348 service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:23.963637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b58380-ccb0-4224-add7-9828863af348 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:23.971657 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b58380-ccb0-4224-add7-9828863af348 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:23.973256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b58380-ccb0-4224-add7-9828863af348 service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:23.983327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1315/2618] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:23:23 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:23:26.814243 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fe905-ccc0-4c18-a2de-a4de98850cf7 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:26.814912 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fe905-ccc0-4c18-a2de-a4de98850cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:26.814912 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fe905-ccc0-4c18-a2de-a4de98850cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:26.834872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-a84fe905-ccc0-4c18-a2de-a4de98850cf7 None None] MFA Rules not processed for user `7ec2605fd1b84e0ea134b943ed33a605`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:26.837595 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a84fe905-ccc0-4c18-a2de-a4de98850cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:26.894182 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1304/2619] 10.222.0.65 () {58 vars in 967 bytes} [Tue May 10 20:23:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:26.907009 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b58380-ccb0-4224-add7-9828863af348 service nova] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:26.908388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-e0b58380-ccb0-4224-add7-9828863af348 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:26.986631 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b58380-ccb0-4224-add7-9828863af348 service nova] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:26.994030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89ef10-317d-47cb-83c6-0262d21a8c1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d10b258a53484da22950e2b1db2e41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:26.994815 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89ef10-317d-47cb-83c6-0262d21a8c1e service swift] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:26.995171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89ef10-317d-47cb-83c6-0262d21a8c1e service swift] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:26.995622 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89ef10-317d-47cb-83c6-0262d21a8c1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:27.004978 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89ef10-317d-47cb-83c6-0262d21a8c1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:27.008346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89ef10-317d-47cb-83c6-0262d21a8c1e service swift] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:27.023031 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1316/2620] 10.222.0.65 () {60 vars in 1384 bytes} [Tue May 10 20:23:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 20:23:27.268782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fe905-ccc0-4c18-a2de-a4de98850cf7 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:27.392135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fe905-ccc0-4c18-a2de-a4de98850cf7 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:27.392135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebf538-2069-40e8-aef5-e55054e4bc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:27.392135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebf538-2069-40e8-aef5-e55054e4bc73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:27.392135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebf538-2069-40e8-aef5-e55054e4bc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:27.392516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebf538-2069-40e8-aef5-e55054e4bc73 admin admin] PATH_INFO: `/v3/users/93b74480c62f4e179089f48b91bd7f28` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:27.392516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ebf538-2069-40e8-aef5-e55054e4bc73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93b74480c62f4e179089f48b91bd7f28)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:27.393031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ebf538-2069-40e8-aef5-e55054e4bc73 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:27.644471 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-b5ebf538-2069-40e8-aef5-e55054e4bc73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93b74480c62f4e179089f48b91bd7f28'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:27.645144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-b5ebf538-2069-40e8-aef5-e55054e4bc73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93b74480c62f4e179089f48b91bd7f28'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:27.657997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-b5ebf538-2069-40e8-aef5-e55054e4bc73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93b74480c62f4e179089f48b91bd7f28'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:27.658698 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-b5ebf538-2069-40e8-aef5-e55054e4bc73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93b74480c62f4e179089f48b91bd7f28'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:27.665458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-b5ebf538-2069-40e8-aef5-e55054e4bc73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93b74480c62f4e179089f48b91bd7f28'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:27.765180 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1305/2621] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:23:27 2022] DELETE /identity/v3/users/93b74480c62f4e179089f48b91bd7f28 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:27.769688 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ca807-b159-4a78-b8e4-552fb9aafaa4 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:27.770192 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ca807-b159-4a78-b8e4-552fb9aafaa4 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:27.770309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ca807-b159-4a78-b8e4-552fb9aafaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:27.788122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-961ca807-b159-4a78-b8e4-552fb9aafaa4 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:27.790339 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-961ca807-b159-4a78-b8e4-552fb9aafaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:27.855901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1317/2622] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:27.863233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebf538-2069-40e8-aef5-e55054e4bc73 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:27.864678 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b5ebf538-2069-40e8-aef5-e55054e4bc73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:27.929046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebf538-2069-40e8-aef5-e55054e4bc73 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:27.932756 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fccda21-3faf-4216-a2c7-ed0480e58344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:27.933418 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fccda21-3faf-4216-a2c7-ed0480e58344 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:27.933739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fccda21-3faf-4216-a2c7-ed0480e58344 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:27.934251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fccda21-3faf-4216-a2c7-ed0480e58344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:27.944419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fccda21-3faf-4216-a2c7-ed0480e58344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:27.945985 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fccda21-3faf-4216-a2c7-ed0480e58344 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:27.958034 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1306/2623] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:23:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:23:28.416635 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ca807-b159-4a78-b8e4-552fb9aafaa4 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:28.419311 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-961ca807-b159-4a78-b8e4-552fb9aafaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:28.447797 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ca807-b159-4a78-b8e4-552fb9aafaa4 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:28.458597 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d86af5-2f0d-4a98-a070-bb5cbdce2b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:28.460195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d86af5-2f0d-4a98-a070-bb5cbdce2b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:28.460489 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d86af5-2f0d-4a98-a070-bb5cbdce2b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:28.460772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d86af5-2f0d-4a98-a070-bb5cbdce2b4b admin admin] PATH_INFO: `/v3/projects/e9bc233a626643f0a6954adc2a215f6c` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:28.462112 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d86af5-2f0d-4a98-a070-bb5cbdce2b4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9bc233a626643f0a6954adc2a215f6c)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:28.472610 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d86af5-2f0d-4a98-a070-bb5cbdce2b4b admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:28.633096 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-84d86af5-2f0d-4a98-a070-bb5cbdce2b4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9bc233a626643f0a6954adc2a215f6c', 'request_id': 'req-84d86af5-2f0d-4a98-a070-bb5cbdce2b4b'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:28.703226 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-84d86af5-2f0d-4a98-a070-bb5cbdce2b4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9bc233a626643f0a6954adc2a215f6c', 'request_id': 'req-84d86af5-2f0d-4a98-a070-bb5cbdce2b4b'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:28.710290 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-84d86af5-2f0d-4a98-a070-bb5cbdce2b4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9bc233a626643f0a6954adc2a215f6c', 'request_id': 'req-84d86af5-2f0d-4a98-a070-bb5cbdce2b4b'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:28.727463 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-84d86af5-2f0d-4a98-a070-bb5cbdce2b4b admin admin] The token cache is being invalidate because project e9bc233a626643f0a6954adc2a215f6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:23:28.727886 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-84d86af5-2f0d-4a98-a070-bb5cbdce2b4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:28.728671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-84d86af5-2f0d-4a98-a070-bb5cbdce2b4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:28.730629 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1318/2624] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:23:28 2022] DELETE /identity/v3/projects/e9bc233a626643f0a6954adc2a215f6c => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:28.735764 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55e092-4a54-4358-be4a-02f08f133c73 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:28.741948 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55e092-4a54-4358-be4a-02f08f133c73 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:28.741948 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55e092-4a54-4358-be4a-02f08f133c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:28.756473 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-db55e092-4a54-4358-be4a-02f08f133c73 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:28.757876 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-db55e092-4a54-4358-be4a-02f08f133c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:28.822486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1307/2625] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:23:28.831336 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d830c9-678c-4463-896d-fc6447bd550e None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:28.831674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d830c9-678c-4463-896d-fc6447bd550e None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:28.831975 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d830c9-678c-4463-896d-fc6447bd550e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:28.850648 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-12d830c9-678c-4463-896d-fc6447bd550e None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:28.851847 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-12d830c9-678c-4463-896d-fc6447bd550e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:28.860392 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1319/2626] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:23:28.864575 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55e092-4a54-4358-be4a-02f08f133c73 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:28.874219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55e092-4a54-4358-be4a-02f08f133c73 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:28.878931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace31a0-ad12-41fe-a339-5e1577192cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:28.879588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace31a0-ad12-41fe-a339-5e1577192cac admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:28.879873 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace31a0-ad12-41fe-a339-5e1577192cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:28.880152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace31a0-ad12-41fe-a339-5e1577192cac admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:28.880892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace31a0-ad12-41fe-a339-5e1577192cac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:23:28.881201 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace31a0-ad12-41fe-a339-5e1577192cac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:28.882858 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace31a0-ad12-41fe-a339-5e1577192cac admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:28.894713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1308/2627] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:23:28.899093 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d830c9-678c-4463-896d-fc6447bd550e None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:28.908980 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d830c9-678c-4463-896d-fc6447bd550e None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:28.913705 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90f68c-7c59-4ab1-9c81-11ccecb39b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:28.914499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90f68c-7c59-4ab1-9c81-11ccecb39b97 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:28.914793 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90f68c-7c59-4ab1-9c81-11ccecb39b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:28.915073 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90f68c-7c59-4ab1-9c81-11ccecb39b97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:28.915813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90f68c-7c59-4ab1-9c81-11ccecb39b97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:28.917777 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90f68c-7c59-4ab1-9c81-11ccecb39b97 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:29.038878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1320/2628] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:23:28 2022] POST /identity/v3/projects => generated 363 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:23:29.042889 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace31a0-ad12-41fe-a339-5e1577192cac admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:29.053871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace31a0-ad12-41fe-a339-5e1577192cac admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:29.125523 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142bbb1-1d13-4dbd-8aad-b593bd536373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:29.126241 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142bbb1-1d13-4dbd-8aad-b593bd536373 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:29.126556 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142bbb1-1d13-4dbd-8aad-b593bd536373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:29.126833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142bbb1-1d13-4dbd-8aad-b593bd536373 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:29.127571 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142bbb1-1d13-4dbd-8aad-b593bd536373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:29.129480 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142bbb1-1d13-4dbd-8aad-b593bd536373 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:29.202544 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1309/2629] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:23:29 2022] POST /identity/v3/users => generated 282 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:23:29.208431 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90f68c-7c59-4ab1-9c81-11ccecb39b97 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:29.214715 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90f68c-7c59-4ab1-9c81-11ccecb39b97 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:29.220283 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c30f90-8e9a-4462-bf0d-ae4fba418982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:29.220938 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c30f90-8e9a-4462-bf0d-ae4fba418982 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:29.221218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c30f90-8e9a-4462-bf0d-ae4fba418982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:29.221489 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c30f90-8e9a-4462-bf0d-ae4fba418982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:29.222252 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c30f90-8e9a-4462-bf0d-ae4fba418982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:29.223828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c30f90-8e9a-4462-bf0d-ae4fba418982 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:29.234707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1321/2630] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:23:29.239569 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142bbb1-1d13-4dbd-8aad-b593bd536373 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:29.249850 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142bbb1-1d13-4dbd-8aad-b593bd536373 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:29.254566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd633f-d19e-46ea-bf42-fb55b6c064e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:29.255226 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd633f-d19e-46ea-bf42-fb55b6c064e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:29.255595 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd633f-d19e-46ea-bf42-fb55b6c064e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:29.256066 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd633f-d19e-46ea-bf42-fb55b6c064e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:29.256810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd633f-d19e-46ea-bf42-fb55b6c064e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:29.258560 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd633f-d19e-46ea-bf42-fb55b6c064e7 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:29.269330 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1310/2631] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:23:29.273520 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c30f90-8e9a-4462-bf0d-ae4fba418982 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:29.284209 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c30f90-8e9a-4462-bf0d-ae4fba418982 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:29.289094 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c8b09-8645-4fc0-84cb-0670659eaf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:29.289966 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c8b09-8645-4fc0-84cb-0670659eaf86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:29.290285 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c8b09-8645-4fc0-84cb-0670659eaf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:29.290564 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c8b09-8645-4fc0-84cb-0670659eaf86 admin admin] PATH_INFO: `/v3/projects/3001d397f6b24230be78467a3c051003/users/43ce6f12de754042b082d11ce1b8ec11/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:29.304687 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c8b09-8645-4fc0-84cb-0670659eaf86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3001d397f6b24230be78467a3c051003, user_id=43ce6f12de754042b082d11ce1b8ec11, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:29.307379 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c8b09-8645-4fc0-84cb-0670659eaf86 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:29.427964 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1322/2632] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:23:29 2022] PUT /identity/v3/projects/3001d397f6b24230be78467a3c051003/users/43ce6f12de754042b082d11ce1b8ec11/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:29.438976 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31909b-88d8-4a27-9204-9a9d0d22d5f5 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:29.439270 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31909b-88d8-4a27-9204-9a9d0d22d5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:29.439547 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31909b-88d8-4a27-9204-9a9d0d22d5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:29.465324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-ee31909b-88d8-4a27-9204-9a9d0d22d5f5 None None] MFA Rules not processed for user `43ce6f12de754042b082d11ce1b8ec11`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:29.467165 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-ee31909b-88d8-4a27-9204-9a9d0d22d5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:29.531338 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1311/2633] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:29 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:29.535872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c8b09-8645-4fc0-84cb-0670659eaf86 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:29.596550 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c8b09-8645-4fc0-84cb-0670659eaf86 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:29.601375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195b9c4-0df8-421a-9b0d-58b43388c6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:29.602029 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195b9c4-0df8-421a-9b0d-58b43388c6b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:29.602346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195b9c4-0df8-421a-9b0d-58b43388c6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:29.602620 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195b9c4-0df8-421a-9b0d-58b43388c6b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:29.603315 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195b9c4-0df8-421a-9b0d-58b43388c6b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:29.605266 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195b9c4-0df8-421a-9b0d-58b43388c6b5 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:29.729363 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1323/2634] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:23:29 2022] POST /identity/v3/projects => generated 363 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:23:29.733485 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31909b-88d8-4a27-9204-9a9d0d22d5f5 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:29.744150 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31909b-88d8-4a27-9204-9a9d0d22d5f5 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:29.798341 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad49d8-f968-4cf3-823e-2cfb0795e126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:29.801088 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad49d8-f968-4cf3-823e-2cfb0795e126 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:29.801088 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad49d8-f968-4cf3-823e-2cfb0795e126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:29.801088 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad49d8-f968-4cf3-823e-2cfb0795e126 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:29.801088 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad49d8-f968-4cf3-823e-2cfb0795e126 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:29.802199 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad49d8-f968-4cf3-823e-2cfb0795e126 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:29.922512 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1312/2635] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:23:29 2022] POST /identity/v3/users => generated 288 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:23:29.923736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195b9c4-0df8-421a-9b0d-58b43388c6b5 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:29.933810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195b9c4-0df8-421a-9b0d-58b43388c6b5 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:29.939856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a9a7d-8dfe-471f-98a5-8a5f59822c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:29.940517 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a9a7d-8dfe-471f-98a5-8a5f59822c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:29.940795 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a9a7d-8dfe-471f-98a5-8a5f59822c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:29.941071 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a9a7d-8dfe-471f-98a5-8a5f59822c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:29.941861 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a9a7d-8dfe-471f-98a5-8a5f59822c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:29.943481 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a9a7d-8dfe-471f-98a5-8a5f59822c9b admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:29.954594 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1324/2636] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:23:29.958470 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad49d8-f968-4cf3-823e-2cfb0795e126 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:29.968475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad49d8-f968-4cf3-823e-2cfb0795e126 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:29.973077 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05530fd-f55a-4fd7-9b18-39957098069e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:29.973960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05530fd-f55a-4fd7-9b18-39957098069e admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:29.974277 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05530fd-f55a-4fd7-9b18-39957098069e admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:29.974557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05530fd-f55a-4fd7-9b18-39957098069e admin admin] PATH_INFO: `/v3/projects/ea7c0fb9dd9a4124946abb5ac95a6123/users/312bbfc97d8144a584f4fc16cf214abd/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:29.990109 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05530fd-f55a-4fd7-9b18-39957098069e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea7c0fb9dd9a4124946abb5ac95a6123, user_id=312bbfc97d8144a584f4fc16cf214abd, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:29.992680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05530fd-f55a-4fd7-9b18-39957098069e admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:30.050859 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1313/2637] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:23:29 2022] PUT /identity/v3/projects/ea7c0fb9dd9a4124946abb5ac95a6123/users/312bbfc97d8144a584f4fc16cf214abd/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:30.055081 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a9a7d-8dfe-471f-98a5-8a5f59822c9b admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:30.065256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a9a7d-8dfe-471f-98a5-8a5f59822c9b admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:30.120955 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0c0f9-0988-4e0d-887b-0ce705383ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:30.121686 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0c0f9-0988-4e0d-887b-0ce705383ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:30.121977 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0c0f9-0988-4e0d-887b-0ce705383ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:30.122284 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0c0f9-0988-4e0d-887b-0ce705383ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:30.123619 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0c0f9-0988-4e0d-887b-0ce705383ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:30.125215 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0c0f9-0988-4e0d-887b-0ce705383ab3 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:30.134987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1325/2638] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:23:30.139761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05530fd-f55a-4fd7-9b18-39957098069e admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:30.150014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05530fd-f55a-4fd7-9b18-39957098069e admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:30.155255 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d60f8-5559-4060-8c1b-c036f4433195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:30.156133 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d60f8-5559-4060-8c1b-c036f4433195 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:30.156539 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d60f8-5559-4060-8c1b-c036f4433195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:30.156821 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d60f8-5559-4060-8c1b-c036f4433195 admin admin] PATH_INFO: `/v3/projects/ea7c0fb9dd9a4124946abb5ac95a6123/users/312bbfc97d8144a584f4fc16cf214abd/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:30.158841 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d60f8-5559-4060-8c1b-c036f4433195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea7c0fb9dd9a4124946abb5ac95a6123, user_id=312bbfc97d8144a584f4fc16cf214abd, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:30.161548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d60f8-5559-4060-8c1b-c036f4433195 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:30.172453 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1314/2639] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:23:30 2022] PUT /identity/v3/projects/ea7c0fb9dd9a4124946abb5ac95a6123/users/312bbfc97d8144a584f4fc16cf214abd/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:30.183956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ac50a-087d-4405-a9c4-9122d4c051dc None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:30.184249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ac50a-087d-4405-a9c4-9122d4c051dc None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:30.184531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ac50a-087d-4405-a9c4-9122d4c051dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:30.213276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-2b3ac50a-087d-4405-a9c4-9122d4c051dc None None] MFA Rules not processed for user `312bbfc97d8144a584f4fc16cf214abd`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:30.214670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-2b3ac50a-087d-4405-a9c4-9122d4c051dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:30.281250 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1326/2640] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:30 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:30.292343 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d60f8-5559-4060-8c1b-c036f4433195 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:30.294009 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-928d60f8-5559-4060-8c1b-c036f4433195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:30.350826 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d60f8-5559-4060-8c1b-c036f4433195 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:30.354517 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39659a4b-1563-42ef-a5e1-c6d01b982c0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:30.355158 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39659a4b-1563-42ef-a5e1-c6d01b982c0e service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:30.355444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39659a4b-1563-42ef-a5e1-c6d01b982c0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:30.355722 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39659a4b-1563-42ef-a5e1-c6d01b982c0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:30.365443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39659a4b-1563-42ef-a5e1-c6d01b982c0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:30.366894 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39659a4b-1563-42ef-a5e1-c6d01b982c0e service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:30.376690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1315/2641] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:23:30 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:23:30.541781 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ac50a-087d-4405-a9c4-9122d4c051dc None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:30.551452 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ac50a-087d-4405-a9c4-9122d4c051dc None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:30.555065 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778fc76-fd01-4fb4-8991-af5d4c6484de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:30.555703 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778fc76-fd01-4fb4-8991-af5d4c6484de service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:30.555990 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778fc76-fd01-4fb4-8991-af5d4c6484de service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:30.556269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778fc76-fd01-4fb4-8991-af5d4c6484de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:30.612787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778fc76-fd01-4fb4-8991-af5d4c6484de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:30.614254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778fc76-fd01-4fb4-8991-af5d4c6484de service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:30.623539 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1327/2642] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:23:30 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:23:42.369475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39659a4b-1563-42ef-a5e1-c6d01b982c0e service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:42.436302 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39659a4b-1563-42ef-a5e1-c6d01b982c0e service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:42.441582 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb928ab7-e741-41a3-a588-65155e40d813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:42.442513 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb928ab7-e741-41a3-a588-65155e40d813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:42.442926 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb928ab7-e741-41a3-a588-65155e40d813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:42.443237 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb928ab7-e741-41a3-a588-65155e40d813 admin admin] PATH_INFO: `/v3/users/43ce6f12de754042b082d11ce1b8ec11` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:42.444440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb928ab7-e741-41a3-a588-65155e40d813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43ce6f12de754042b082d11ce1b8ec11)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:42.446419 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb928ab7-e741-41a3-a588-65155e40d813 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:42.607832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-cb928ab7-e741-41a3-a588-65155e40d813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43ce6f12de754042b082d11ce1b8ec11'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:42.609447 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-cb928ab7-e741-41a3-a588-65155e40d813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43ce6f12de754042b082d11ce1b8ec11'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:42.621621 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-cb928ab7-e741-41a3-a588-65155e40d813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43ce6f12de754042b082d11ce1b8ec11'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:42.622424 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-cb928ab7-e741-41a3-a588-65155e40d813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43ce6f12de754042b082d11ce1b8ec11'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:42.630039 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-cb928ab7-e741-41a3-a588-65155e40d813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43ce6f12de754042b082d11ce1b8ec11'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:42.737517 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1316/2643] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:23:42 2022] DELETE /identity/v3/users/43ce6f12de754042b082d11ce1b8ec11 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:42.742099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89603abc-08d2-4cad-b13e-e48565b48e67 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:42.742579 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89603abc-08d2-4cad-b13e-e48565b48e67 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:42.742677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89603abc-08d2-4cad-b13e-e48565b48e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:42.760421 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-89603abc-08d2-4cad-b13e-e48565b48e67 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:42.761908 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-89603abc-08d2-4cad-b13e-e48565b48e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:42.824620 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1328/2644] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:42.835418 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb928ab7-e741-41a3-a588-65155e40d813 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:42.836829 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-cb928ab7-e741-41a3-a588-65155e40d813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:42.897842 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb928ab7-e741-41a3-a588-65155e40d813 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:42.901531 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f254d1-8a89-4d9c-940c-fc0216d2bc37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:42.902304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f254d1-8a89-4d9c-940c-fc0216d2bc37 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:42.902590 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f254d1-8a89-4d9c-940c-fc0216d2bc37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:42.902865 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f254d1-8a89-4d9c-940c-fc0216d2bc37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:42.916756 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f254d1-8a89-4d9c-940c-fc0216d2bc37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:42.918295 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f254d1-8a89-4d9c-940c-fc0216d2bc37 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:42.928466 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1317/2645] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:23:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:23:43.315991 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89603abc-08d2-4cad-b13e-e48565b48e67 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:43.317696 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-89603abc-08d2-4cad-b13e-e48565b48e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:43.334599 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89603abc-08d2-4cad-b13e-e48565b48e67 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:43.346534 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b42a80-d0cf-4946-a250-edd09d59f820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:43.347506 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b42a80-d0cf-4946-a250-edd09d59f820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:43.347956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b42a80-d0cf-4946-a250-edd09d59f820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:43.348307 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b42a80-d0cf-4946-a250-edd09d59f820 admin admin] PATH_INFO: `/v3/projects/3001d397f6b24230be78467a3c051003` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:43.350349 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b42a80-d0cf-4946-a250-edd09d59f820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3001d397f6b24230be78467a3c051003)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:43.366142 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b42a80-d0cf-4946-a250-edd09d59f820 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:43.514203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e7b42a80-d0cf-4946-a250-edd09d59f820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3001d397f6b24230be78467a3c051003', 'request_id': 'req-e7b42a80-d0cf-4946-a250-edd09d59f820'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:43.576288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e7b42a80-d0cf-4946-a250-edd09d59f820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3001d397f6b24230be78467a3c051003', 'request_id': 'req-e7b42a80-d0cf-4946-a250-edd09d59f820'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:43.583022 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e7b42a80-d0cf-4946-a250-edd09d59f820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3001d397f6b24230be78467a3c051003', 'request_id': 'req-e7b42a80-d0cf-4946-a250-edd09d59f820'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:43.595117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e7b42a80-d0cf-4946-a250-edd09d59f820 admin admin] The token cache is being invalidate because project 3001d397f6b24230be78467a3c051003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:23:43.595335 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e7b42a80-d0cf-4946-a250-edd09d59f820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:43.595990 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-e7b42a80-d0cf-4946-a250-edd09d59f820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:43.597550 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1329/2646] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:23:43 2022] DELETE /identity/v3/projects/3001d397f6b24230be78467a3c051003 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:43.601651 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f254d1-8a89-4d9c-940c-fc0216d2bc37 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:43.614632 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f254d1-8a89-4d9c-940c-fc0216d2bc37 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:43.670113 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073aee1c-a066-43c1-9d48-93f8c63ede41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:43.671002 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073aee1c-a066-43c1-9d48-93f8c63ede41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:43.671824 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073aee1c-a066-43c1-9d48-93f8c63ede41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:43.672102 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073aee1c-a066-43c1-9d48-93f8c63ede41 admin admin] PATH_INFO: `/v3/users/312bbfc97d8144a584f4fc16cf214abd` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:43.673346 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073aee1c-a066-43c1-9d48-93f8c63ede41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=312bbfc97d8144a584f4fc16cf214abd)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:43.675288 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073aee1c-a066-43c1-9d48-93f8c63ede41 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:43.869918 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-073aee1c-a066-43c1-9d48-93f8c63ede41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '312bbfc97d8144a584f4fc16cf214abd'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:43.870729 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-073aee1c-a066-43c1-9d48-93f8c63ede41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '312bbfc97d8144a584f4fc16cf214abd'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:43.887276 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-073aee1c-a066-43c1-9d48-93f8c63ede41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '312bbfc97d8144a584f4fc16cf214abd'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:43.888010 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-073aee1c-a066-43c1-9d48-93f8c63ede41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '312bbfc97d8144a584f4fc16cf214abd'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:43.895678 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-073aee1c-a066-43c1-9d48-93f8c63ede41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '312bbfc97d8144a584f4fc16cf214abd'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:43.971161 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1318/2647] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:23:43 2022] DELETE /identity/v3/users/312bbfc97d8144a584f4fc16cf214abd => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:44.252238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b42a80-d0cf-4946-a250-edd09d59f820 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:44.253794 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-e7b42a80-d0cf-4946-a250-edd09d59f820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:44.332191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b42a80-d0cf-4946-a250-edd09d59f820 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:44.396346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497a137-43c6-40b3-a337-eb319f362124 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:44.396927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497a137-43c6-40b3-a337-eb319f362124 service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:44.397215 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497a137-43c6-40b3-a337-eb319f362124 service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:44.397496 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497a137-43c6-40b3-a337-eb319f362124 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:44.399358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7497a137-43c6-40b3-a337-eb319f362124 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:44.411600 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7497a137-43c6-40b3-a337-eb319f362124 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:44.412307 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073aee1c-a066-43c1-9d48-93f8c63ede41 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:44.413205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7497a137-43c6-40b3-a337-eb319f362124 service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:44.414914 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-073aee1c-a066-43c1-9d48-93f8c63ede41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:44.425879 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1330/2648] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:23:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:23:44.516360 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073aee1c-a066-43c1-9d48-93f8c63ede41 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:44.521059 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5e9d4-800a-4098-ba39-87cd9a8be0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:44.521929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5e9d4-800a-4098-ba39-87cd9a8be0ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:44.522257 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5e9d4-800a-4098-ba39-87cd9a8be0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:44.522941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5e9d4-800a-4098-ba39-87cd9a8be0ea admin admin] PATH_INFO: `/v3/projects/ea7c0fb9dd9a4124946abb5ac95a6123` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:44.524181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5e9d4-800a-4098-ba39-87cd9a8be0ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea7c0fb9dd9a4124946abb5ac95a6123)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:44.526121 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5e9d4-800a-4098-ba39-87cd9a8be0ea admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:44.646721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f9c5e9d4-800a-4098-ba39-87cd9a8be0ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea7c0fb9dd9a4124946abb5ac95a6123', 'request_id': 'req-f9c5e9d4-800a-4098-ba39-87cd9a8be0ea'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:44.708416 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f9c5e9d4-800a-4098-ba39-87cd9a8be0ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea7c0fb9dd9a4124946abb5ac95a6123', 'request_id': 'req-f9c5e9d4-800a-4098-ba39-87cd9a8be0ea'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:44.716069 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f9c5e9d4-800a-4098-ba39-87cd9a8be0ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea7c0fb9dd9a4124946abb5ac95a6123', 'request_id': 'req-f9c5e9d4-800a-4098-ba39-87cd9a8be0ea'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:44.730710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f9c5e9d4-800a-4098-ba39-87cd9a8be0ea admin admin] The token cache is being invalidate because project ea7c0fb9dd9a4124946abb5ac95a6123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:23:44.730993 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f9c5e9d4-800a-4098-ba39-87cd9a8be0ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:44.731693 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-f9c5e9d4-800a-4098-ba39-87cd9a8be0ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:44.737248 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1319/2649] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:23:44 2022] DELETE /identity/v3/projects/ea7c0fb9dd9a4124946abb5ac95a6123 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:44.739177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34432daf-e26d-4e46-b4f4-723f8fd20716 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:44.739720 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34432daf-e26d-4e46-b4f4-723f8fd20716 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:44.740005 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34432daf-e26d-4e46-b4f4-723f8fd20716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:44.762453 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-34432daf-e26d-4e46-b4f4-723f8fd20716 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:44.763769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-34432daf-e26d-4e46-b4f4-723f8fd20716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:44.840952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1331/2650] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:23:44.849483 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f21531-ca9d-49f6-99d6-367c5e6994ae None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:44.849917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f21531-ca9d-49f6-99d6-367c5e6994ae None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:44.850319 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f21531-ca9d-49f6-99d6-367c5e6994ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:44.869997 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-a5f21531-ca9d-49f6-99d6-367c5e6994ae None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:44.871233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a5f21531-ca9d-49f6-99d6-367c5e6994ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:44.885071 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1320/2651] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:23:44.890926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34432daf-e26d-4e46-b4f4-723f8fd20716 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:44.902125 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34432daf-e26d-4e46-b4f4-723f8fd20716 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:44.908557 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db917ca3-e468-48c5-bc15-3fedb4d15717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:44.909253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db917ca3-e468-48c5-bc15-3fedb4d15717 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:44.909680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db917ca3-e468-48c5-bc15-3fedb4d15717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:44.909966 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db917ca3-e468-48c5-bc15-3fedb4d15717 admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:44.910748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db917ca3-e468-48c5-bc15-3fedb4d15717 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:23:44.911059 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db917ca3-e468-48c5-bc15-3fedb4d15717 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:44.912792 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db917ca3-e468-48c5-bc15-3fedb4d15717 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:44.930916 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1332/2652] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:23:44.932581 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f21531-ca9d-49f6-99d6-367c5e6994ae None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:44.945224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f21531-ca9d-49f6-99d6-367c5e6994ae None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:44.950886 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d8d08-1586-4c6c-a2b1-0f83c335c185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:44.951624 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d8d08-1586-4c6c-a2b1-0f83c335c185 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:44.951980 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d8d08-1586-4c6c-a2b1-0f83c335c185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:44.952324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d8d08-1586-4c6c-a2b1-0f83c335c185 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:44.953144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d8d08-1586-4c6c-a2b1-0f83c335c185 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:44.955465 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d8d08-1586-4c6c-a2b1-0f83c335c185 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:45.076557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1321/2653] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:23:44 2022] POST /identity/v3/projects => generated 381 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:23:45.078340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db917ca3-e468-48c5-bc15-3fedb4d15717 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:45.090542 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db917ca3-e468-48c5-bc15-3fedb4d15717 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:45.160901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbf728-b330-4b0e-ad24-a6db8dfbb230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:45.161488 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbf728-b330-4b0e-ad24-a6db8dfbb230 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:45.161771 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbf728-b330-4b0e-ad24-a6db8dfbb230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:45.162052 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbf728-b330-4b0e-ad24-a6db8dfbb230 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:45.162774 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbf728-b330-4b0e-ad24-a6db8dfbb230 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:45.164676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbf728-b330-4b0e-ad24-a6db8dfbb230 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:45.248894 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1333/2654] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:23:45 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:23:45.253751 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d8d08-1586-4c6c-a2b1-0f83c335c185 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:45.266247 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d8d08-1586-4c6c-a2b1-0f83c335c185 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:45.271335 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83270f-9ffb-445c-88fd-1b25a12f53bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:45.271959 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83270f-9ffb-445c-88fd-1b25a12f53bb admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:45.272342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83270f-9ffb-445c-88fd-1b25a12f53bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:45.272626 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83270f-9ffb-445c-88fd-1b25a12f53bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:45.273369 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83270f-9ffb-445c-88fd-1b25a12f53bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:45.275009 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83270f-9ffb-445c-88fd-1b25a12f53bb admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:45.310005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1322/2655] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:23:45.314625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbf728-b330-4b0e-ad24-a6db8dfbb230 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:45.325023 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbf728-b330-4b0e-ad24-a6db8dfbb230 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:45.331551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae580e55-8e13-4a06-9523-a3ea817e42d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:45.332287 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae580e55-8e13-4a06-9523-a3ea817e42d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:45.333459 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae580e55-8e13-4a06-9523-a3ea817e42d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:45.333459 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae580e55-8e13-4a06-9523-a3ea817e42d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:45.333669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae580e55-8e13-4a06-9523-a3ea817e42d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:45.335223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae580e55-8e13-4a06-9523-a3ea817e42d1 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:45.346289 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1334/2656] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:23:45.350342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83270f-9ffb-445c-88fd-1b25a12f53bb admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:45.361563 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83270f-9ffb-445c-88fd-1b25a12f53bb admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:45.366414 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fa3fe-941b-4181-b1a8-57d53cb530ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:45.367366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fa3fe-941b-4181-b1a8-57d53cb530ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:45.367658 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fa3fe-941b-4181-b1a8-57d53cb530ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:45.367934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fa3fe-941b-4181-b1a8-57d53cb530ea admin admin] PATH_INFO: `/v3/projects/2a5572fa7def4b1aa9268978576aeb30/users/2ec7055930d940539e23bb4bc3a9c8d4/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:45.383825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fa3fe-941b-4181-b1a8-57d53cb530ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a5572fa7def4b1aa9268978576aeb30, user_id=2ec7055930d940539e23bb4bc3a9c8d4, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:45.386512 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fa3fe-941b-4181-b1a8-57d53cb530ea admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:45.457553 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1323/2657] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:23:45 2022] PUT /identity/v3/projects/2a5572fa7def4b1aa9268978576aeb30/users/2ec7055930d940539e23bb4bc3a9c8d4/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:45.467965 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cd303-c12d-46d3-9e0a-f955e023b0b0 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:45.468873 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cd303-c12d-46d3-9e0a-f955e023b0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:45.469153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cd303-c12d-46d3-9e0a-f955e023b0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:45.495199 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-3b5cd303-c12d-46d3-9e0a-f955e023b0b0 None None] MFA Rules not processed for user `2ec7055930d940539e23bb4bc3a9c8d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:45.496375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3b5cd303-c12d-46d3-9e0a-f955e023b0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:45.561611 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1335/2658] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:45.566150 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fa3fe-941b-4181-b1a8-57d53cb530ea admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:45.626107 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fa3fe-941b-4181-b1a8-57d53cb530ea admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:45.630768 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb8714-e61d-4c07-9e26-c0fc835dd692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:45.631426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb8714-e61d-4c07-9e26-c0fc835dd692 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:45.631713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb8714-e61d-4c07-9e26-c0fc835dd692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:45.631993 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb8714-e61d-4c07-9e26-c0fc835dd692 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:45.632683 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb8714-e61d-4c07-9e26-c0fc835dd692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:45.634626 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb8714-e61d-4c07-9e26-c0fc835dd692 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:45.723162 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1324/2659] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:23:45 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:23:45.727247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cd303-c12d-46d3-9e0a-f955e023b0b0 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:45.738546 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cd303-c12d-46d3-9e0a-f955e023b0b0 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:45.793048 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace07c92-81a0-4630-96a3-660aee1ef0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:45.793708 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace07c92-81a0-4630-96a3-660aee1ef0ad admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:45.793993 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace07c92-81a0-4630-96a3-660aee1ef0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:45.794302 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace07c92-81a0-4630-96a3-660aee1ef0ad admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:45.794992 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace07c92-81a0-4630-96a3-660aee1ef0ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:45.797514 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace07c92-81a0-4630-96a3-660aee1ef0ad admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:45.876926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1336/2660] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:23:45 2022] POST /identity/v3/users => generated 297 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:23:45.881134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb8714-e61d-4c07-9e26-c0fc835dd692 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:45.892327 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb8714-e61d-4c07-9e26-c0fc835dd692 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:45.896889 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff656830-8757-4c45-9c98-d3b259283e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:45.897536 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff656830-8757-4c45-9c98-d3b259283e8d admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:45.897817 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff656830-8757-4c45-9c98-d3b259283e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:45.898251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff656830-8757-4c45-9c98-d3b259283e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:45.898978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff656830-8757-4c45-9c98-d3b259283e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:45.900565 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff656830-8757-4c45-9c98-d3b259283e8d admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:45.911478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1325/2661] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:23:45.915613 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace07c92-81a0-4630-96a3-660aee1ef0ad admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:45.926149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace07c92-81a0-4630-96a3-660aee1ef0ad admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:45.930778 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64ebbc-64e1-47ff-a604-2a4ea96be8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:45.931654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64ebbc-64e1-47ff-a604-2a4ea96be8f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:45.931942 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64ebbc-64e1-47ff-a604-2a4ea96be8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:45.932226 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64ebbc-64e1-47ff-a604-2a4ea96be8f4 admin admin] PATH_INFO: `/v3/projects/68f886255d4c41faa5e987d976503229/users/37acbd4c1b9d4843998b1200303897d4/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:45.948108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64ebbc-64e1-47ff-a604-2a4ea96be8f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f886255d4c41faa5e987d976503229, user_id=37acbd4c1b9d4843998b1200303897d4, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:45.950817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64ebbc-64e1-47ff-a604-2a4ea96be8f4 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:46.016547 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1337/2662] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:23:45 2022] PUT /identity/v3/projects/68f886255d4c41faa5e987d976503229/users/37acbd4c1b9d4843998b1200303897d4/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:46.020521 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff656830-8757-4c45-9c98-d3b259283e8d admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:46.030510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff656830-8757-4c45-9c98-d3b259283e8d admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:46.085932 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9c28e-ca53-4a00-b951-72dece4c3828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:46.086619 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9c28e-ca53-4a00-b951-72dece4c3828 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:46.086902 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9c28e-ca53-4a00-b951-72dece4c3828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:46.087178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9c28e-ca53-4a00-b951-72dece4c3828 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:46.087904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9c28e-ca53-4a00-b951-72dece4c3828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:46.089598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9c28e-ca53-4a00-b951-72dece4c3828 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:46.099046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1326/2663] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:23:46.103086 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64ebbc-64e1-47ff-a604-2a4ea96be8f4 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:46.113158 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64ebbc-64e1-47ff-a604-2a4ea96be8f4 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:46.117745 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ca6bd-5d34-44cf-ad71-aed996c54a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:46.118653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ca6bd-5d34-44cf-ad71-aed996c54a99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:46.118939 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ca6bd-5d34-44cf-ad71-aed996c54a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:46.119219 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ca6bd-5d34-44cf-ad71-aed996c54a99 admin admin] PATH_INFO: `/v3/projects/68f886255d4c41faa5e987d976503229/users/37acbd4c1b9d4843998b1200303897d4/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:46.121195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ca6bd-5d34-44cf-ad71-aed996c54a99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f886255d4c41faa5e987d976503229, user_id=37acbd4c1b9d4843998b1200303897d4, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:46.123849 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ca6bd-5d34-44cf-ad71-aed996c54a99 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:46.134362 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1338/2664] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:23:46 2022] PUT /identity/v3/projects/68f886255d4c41faa5e987d976503229/users/37acbd4c1b9d4843998b1200303897d4/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:46.145821 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbf3eb-5474-466b-aac9-eb47d2c00f04 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:46.146131 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbf3eb-5474-466b-aac9-eb47d2c00f04 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:46.146415 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbf3eb-5474-466b-aac9-eb47d2c00f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:46.172776 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-ffbbf3eb-5474-466b-aac9-eb47d2c00f04 None None] MFA Rules not processed for user `37acbd4c1b9d4843998b1200303897d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:46.173985 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-ffbbf3eb-5474-466b-aac9-eb47d2c00f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:46.241919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1327/2665] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:23:46.253196 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ca6bd-5d34-44cf-ad71-aed996c54a99 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:46.254882 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4e2ca6bd-5d34-44cf-ad71-aed996c54a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:46.310608 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ca6bd-5d34-44cf-ad71-aed996c54a99 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:46.314706 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468259a-e939-4f89-8cd6-7640fc9d2e20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:46.315339 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468259a-e939-4f89-8cd6-7640fc9d2e20 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:46.315620 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468259a-e939-4f89-8cd6-7640fc9d2e20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:46.315892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468259a-e939-4f89-8cd6-7640fc9d2e20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:46.324885 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a468259a-e939-4f89-8cd6-7640fc9d2e20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:46.326369 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a468259a-e939-4f89-8cd6-7640fc9d2e20 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:46.337415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1339/2666] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:23:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:23:46.838445 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbf3eb-5474-466b-aac9-eb47d2c00f04 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:46.901499 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbf3eb-5474-466b-aac9-eb47d2c00f04 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:46.906026 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae25db-d009-405b-b2d6-7e7eddc00136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:46.906925 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae25db-d009-405b-b2d6-7e7eddc00136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:46.907207 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae25db-d009-405b-b2d6-7e7eddc00136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:46.907482 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae25db-d009-405b-b2d6-7e7eddc00136 admin admin] PATH_INFO: `/v3/users/2ec7055930d940539e23bb4bc3a9c8d4` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:46.908673 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae25db-d009-405b-b2d6-7e7eddc00136 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ec7055930d940539e23bb4bc3a9c8d4)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:46.910619 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae25db-d009-405b-b2d6-7e7eddc00136 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:47.074856 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-85ae25db-d009-405b-b2d6-7e7eddc00136 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ec7055930d940539e23bb4bc3a9c8d4'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:47.075429 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-85ae25db-d009-405b-b2d6-7e7eddc00136 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ec7055930d940539e23bb4bc3a9c8d4'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:47.087006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-85ae25db-d009-405b-b2d6-7e7eddc00136 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ec7055930d940539e23bb4bc3a9c8d4'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:47.087632 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-85ae25db-d009-405b-b2d6-7e7eddc00136 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ec7055930d940539e23bb4bc3a9c8d4'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:47.094401 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-85ae25db-d009-405b-b2d6-7e7eddc00136 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ec7055930d940539e23bb4bc3a9c8d4'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:47.169920 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1328/2667] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:23:46 2022] DELETE /identity/v3/users/2ec7055930d940539e23bb4bc3a9c8d4 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:47.174533 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b3ea8-fc71-4e13-82ba-ea0ddc6fd114 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:47.174828 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b3ea8-fc71-4e13-82ba-ea0ddc6fd114 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:47.175158 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b3ea8-fc71-4e13-82ba-ea0ddc6fd114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:47.192303 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-6b7b3ea8-fc71-4e13-82ba-ea0ddc6fd114 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:47.193505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-6b7b3ea8-fc71-4e13-82ba-ea0ddc6fd114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:47.251449 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1340/2668] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:47.260964 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae25db-d009-405b-b2d6-7e7eddc00136 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:47.262401 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-85ae25db-d009-405b-b2d6-7e7eddc00136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:47.323680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae25db-d009-405b-b2d6-7e7eddc00136 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:47.327475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715a192-8db2-47b2-8dcd-97de2a90fa5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:47.328060 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715a192-8db2-47b2-8dcd-97de2a90fa5f service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:47.328350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715a192-8db2-47b2-8dcd-97de2a90fa5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:47.329166 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715a192-8db2-47b2-8dcd-97de2a90fa5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:47.338441 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715a192-8db2-47b2-8dcd-97de2a90fa5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:47.339869 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715a192-8db2-47b2-8dcd-97de2a90fa5f service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:47.349836 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1329/2669] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:23:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:23:47.736126 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b3ea8-fc71-4e13-82ba-ea0ddc6fd114 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:47.738238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-6b7b3ea8-fc71-4e13-82ba-ea0ddc6fd114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:47.752572 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b3ea8-fc71-4e13-82ba-ea0ddc6fd114 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:47.757694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57c086-61d3-4a5a-ad69-c88ba0495fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:47.758743 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57c086-61d3-4a5a-ad69-c88ba0495fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:47.759098 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57c086-61d3-4a5a-ad69-c88ba0495fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:47.759280 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57c086-61d3-4a5a-ad69-c88ba0495fd4 admin admin] PATH_INFO: `/v3/projects/2a5572fa7def4b1aa9268978576aeb30` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:47.760463 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57c086-61d3-4a5a-ad69-c88ba0495fd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a5572fa7def4b1aa9268978576aeb30)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:47.762390 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57c086-61d3-4a5a-ad69-c88ba0495fd4 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:47.867698 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8d57c086-61d3-4a5a-ad69-c88ba0495fd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a5572fa7def4b1aa9268978576aeb30', 'request_id': 'req-8d57c086-61d3-4a5a-ad69-c88ba0495fd4'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:48.059926 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8d57c086-61d3-4a5a-ad69-c88ba0495fd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a5572fa7def4b1aa9268978576aeb30', 'request_id': 'req-8d57c086-61d3-4a5a-ad69-c88ba0495fd4'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:48.066694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8d57c086-61d3-4a5a-ad69-c88ba0495fd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a5572fa7def4b1aa9268978576aeb30', 'request_id': 'req-8d57c086-61d3-4a5a-ad69-c88ba0495fd4'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:48.077789 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8d57c086-61d3-4a5a-ad69-c88ba0495fd4 admin admin] The token cache is being invalidate because project 2a5572fa7def4b1aa9268978576aeb30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:23:48.078099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8d57c086-61d3-4a5a-ad69-c88ba0495fd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:48.079266 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-8d57c086-61d3-4a5a-ad69-c88ba0495fd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:48.081075 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1341/2670] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:23:47 2022] DELETE /identity/v3/projects/2a5572fa7def4b1aa9268978576aeb30 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:23:48.085241 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715a192-8db2-47b2-8dcd-97de2a90fa5f service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:48.095401 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715a192-8db2-47b2-8dcd-97de2a90fa5f service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:48.150515 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58b662-c05c-4e83-a3ec-492ebb625f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:48.151295 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58b662-c05c-4e83-a3ec-492ebb625f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:48.151573 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58b662-c05c-4e83-a3ec-492ebb625f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:48.151843 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58b662-c05c-4e83-a3ec-492ebb625f3b admin admin] PATH_INFO: `/v3/users/37acbd4c1b9d4843998b1200303897d4` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:48.153103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58b662-c05c-4e83-a3ec-492ebb625f3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=37acbd4c1b9d4843998b1200303897d4)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:48.155118 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58b662-c05c-4e83-a3ec-492ebb625f3b admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:48.304740 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-bf58b662-c05c-4e83-a3ec-492ebb625f3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37acbd4c1b9d4843998b1200303897d4'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:48.305517 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-bf58b662-c05c-4e83-a3ec-492ebb625f3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37acbd4c1b9d4843998b1200303897d4'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:48.318134 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-bf58b662-c05c-4e83-a3ec-492ebb625f3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37acbd4c1b9d4843998b1200303897d4'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:48.318851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-bf58b662-c05c-4e83-a3ec-492ebb625f3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37acbd4c1b9d4843998b1200303897d4'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:48.325495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-bf58b662-c05c-4e83-a3ec-492ebb625f3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37acbd4c1b9d4843998b1200303897d4'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:48.437009 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1330/2671] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:23:48 2022] DELETE /identity/v3/users/37acbd4c1b9d4843998b1200303897d4 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:23:48.852573 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57c086-61d3-4a5a-ad69-c88ba0495fd4 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:48.856596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-8d57c086-61d3-4a5a-ad69-c88ba0495fd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:48.874983 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57c086-61d3-4a5a-ad69-c88ba0495fd4 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:48.931398 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b390467-ce4c-4117-bf6c-536ec1101ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:48.932333 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b390467-ce4c-4117-bf6c-536ec1101ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:48.932615 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b390467-ce4c-4117-bf6c-536ec1101ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:48.932889 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b390467-ce4c-4117-bf6c-536ec1101ca7 admin admin] PATH_INFO: `/v3/projects/68f886255d4c41faa5e987d976503229` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:48.934091 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b390467-ce4c-4117-bf6c-536ec1101ca7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68f886255d4c41faa5e987d976503229)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:48.936079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b390467-ce4c-4117-bf6c-536ec1101ca7 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:49.050891 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-0b390467-ce4c-4117-bf6c-536ec1101ca7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68f886255d4c41faa5e987d976503229', 'request_id': 'req-0b390467-ce4c-4117-bf6c-536ec1101ca7'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:49.107791 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-0b390467-ce4c-4117-bf6c-536ec1101ca7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68f886255d4c41faa5e987d976503229', 'request_id': 'req-0b390467-ce4c-4117-bf6c-536ec1101ca7'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:49.113961 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-0b390467-ce4c-4117-bf6c-536ec1101ca7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68f886255d4c41faa5e987d976503229', 'request_id': 'req-0b390467-ce4c-4117-bf6c-536ec1101ca7'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:49.125073 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-0b390467-ce4c-4117-bf6c-536ec1101ca7 admin admin] The token cache is being invalidate because project 68f886255d4c41faa5e987d976503229 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:23:49.125958 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-0b390467-ce4c-4117-bf6c-536ec1101ca7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:49.126567 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-0b390467-ce4c-4117-bf6c-536ec1101ca7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:23:49.128163 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1342/2672] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:23:48 2022] DELETE /identity/v3/projects/68f886255d4c41faa5e987d976503229 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:49.133086 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd03f92-a221-4bcc-899c-fe5ded8e5ff4 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:49.133556 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd03f92-a221-4bcc-899c-fe5ded8e5ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:49.133666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd03f92-a221-4bcc-899c-fe5ded8e5ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:49.151591 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-bfd03f92-a221-4bcc-899c-fe5ded8e5ff4 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:49.152699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-bfd03f92-a221-4bcc-899c-fe5ded8e5ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:49.211266 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1331/2673] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:49.219974 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e7746-dd79-4381-8c4f-905d69c810a2 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:49.220342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e7746-dd79-4381-8c4f-905d69c810a2 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:49.220691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e7746-dd79-4381-8c4f-905d69c810a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:49.239251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-a08e7746-dd79-4381-8c4f-905d69c810a2 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:49.240510 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-a08e7746-dd79-4381-8c4f-905d69c810a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:49.249313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1343/2674] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:49.254021 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd03f92-a221-4bcc-899c-fe5ded8e5ff4 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:49.263495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd03f92-a221-4bcc-899c-fe5ded8e5ff4 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:49.269973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899c44d-8219-4a1a-8dac-de81b2dfa90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:49.269973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899c44d-8219-4a1a-8dac-de81b2dfa90a admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:49.269973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899c44d-8219-4a1a-8dac-de81b2dfa90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:49.269973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899c44d-8219-4a1a-8dac-de81b2dfa90a admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:49.270708 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899c44d-8219-4a1a-8dac-de81b2dfa90a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:23:49.271016 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899c44d-8219-4a1a-8dac-de81b2dfa90a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:49.272629 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899c44d-8219-4a1a-8dac-de81b2dfa90a admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:49.284548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1332/2675] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:23:49.289057 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e7746-dd79-4381-8c4f-905d69c810a2 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:49.298875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e7746-dd79-4381-8c4f-905d69c810a2 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:49.303610 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92982f0f-a52a-4b14-a04f-7b7f87d2c19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:49.304293 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92982f0f-a52a-4b14-a04f-7b7f87d2c19e admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:49.304578 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92982f0f-a52a-4b14-a04f-7b7f87d2c19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:49.304854 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92982f0f-a52a-4b14-a04f-7b7f87d2c19e admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:49.305559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92982f0f-a52a-4b14-a04f-7b7f87d2c19e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:49.307480 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92982f0f-a52a-4b14-a04f-7b7f87d2c19e admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:49.393159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1344/2676] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:23:49 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:23:49.397400 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899c44d-8219-4a1a-8dac-de81b2dfa90a admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:49.407919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899c44d-8219-4a1a-8dac-de81b2dfa90a admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:49.468840 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585484a-13e5-4506-b23a-8d291cd51529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:49.469471 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585484a-13e5-4506-b23a-8d291cd51529 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:49.469755 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585484a-13e5-4506-b23a-8d291cd51529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:49.470030 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585484a-13e5-4506-b23a-8d291cd51529 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:49.470993 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585484a-13e5-4506-b23a-8d291cd51529 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:49.472893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585484a-13e5-4506-b23a-8d291cd51529 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:49.562493 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1333/2677] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:23:49 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:23:49.566592 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92982f0f-a52a-4b14-a04f-7b7f87d2c19e admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:49.576737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92982f0f-a52a-4b14-a04f-7b7f87d2c19e admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:49.582162 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f98ed6-cca0-4d91-95e1-57a4c60d73ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:49.584305 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f98ed6-cca0-4d91-95e1-57a4c60d73ec admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:49.584305 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f98ed6-cca0-4d91-95e1-57a4c60d73ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:49.584305 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f98ed6-cca0-4d91-95e1-57a4c60d73ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:49.584305 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f98ed6-cca0-4d91-95e1-57a4c60d73ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:49.585654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f98ed6-cca0-4d91-95e1-57a4c60d73ec admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:49.596422 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1345/2678] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:23:49.600293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585484a-13e5-4506-b23a-8d291cd51529 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:49.610817 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585484a-13e5-4506-b23a-8d291cd51529 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:49.615862 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20c90c-6ac9-4425-bb58-3a0d000ccc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:49.616540 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20c90c-6ac9-4425-bb58-3a0d000ccc91 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:49.616823 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20c90c-6ac9-4425-bb58-3a0d000ccc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:49.617097 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20c90c-6ac9-4425-bb58-3a0d000ccc91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:49.617854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20c90c-6ac9-4425-bb58-3a0d000ccc91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:49.619491 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20c90c-6ac9-4425-bb58-3a0d000ccc91 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:49.629916 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1334/2679] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:23:49.633966 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f98ed6-cca0-4d91-95e1-57a4c60d73ec admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:49.643928 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f98ed6-cca0-4d91-95e1-57a4c60d73ec admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:49.648575 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2bb75-f3ff-4b04-95fb-7e1a6d974f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:49.649440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2bb75-f3ff-4b04-95fb-7e1a6d974f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:49.649725 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2bb75-f3ff-4b04-95fb-7e1a6d974f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:49.650007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2bb75-f3ff-4b04-95fb-7e1a6d974f63 admin admin] PATH_INFO: `/v3/projects/c66479bf2d944babb976591e7ddf569c/users/c5f9c320e64844d7bae7e9a0fe96907f/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:49.664671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2bb75-f3ff-4b04-95fb-7e1a6d974f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c66479bf2d944babb976591e7ddf569c, user_id=c5f9c320e64844d7bae7e9a0fe96907f, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:49.667387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2bb75-f3ff-4b04-95fb-7e1a6d974f63 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:49.753997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1346/2680] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:23:49 2022] PUT /identity/v3/projects/c66479bf2d944babb976591e7ddf569c/users/c5f9c320e64844d7bae7e9a0fe96907f/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:23:49.765727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d6cb7-faf1-4e9a-839f-b08683d5d693 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:49.766050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d6cb7-faf1-4e9a-839f-b08683d5d693 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:49.766389 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d6cb7-faf1-4e9a-839f-b08683d5d693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:49.792823 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-ac7d6cb7-faf1-4e9a-839f-b08683d5d693 None None] MFA Rules not processed for user `c5f9c320e64844d7bae7e9a0fe96907f`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:49.794031 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-ac7d6cb7-faf1-4e9a-839f-b08683d5d693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:49.858010 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1335/2681] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:49.863606 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2bb75-f3ff-4b04-95fb-7e1a6d974f63 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:49.924122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2bb75-f3ff-4b04-95fb-7e1a6d974f63 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:49.928837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13779ab7-c30c-4a81-9bca-c8d1d4dc7826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:49.929487 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13779ab7-c30c-4a81-9bca-c8d1d4dc7826 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:49.929774 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13779ab7-c30c-4a81-9bca-c8d1d4dc7826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:49.930054 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13779ab7-c30c-4a81-9bca-c8d1d4dc7826 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:49.930793 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13779ab7-c30c-4a81-9bca-c8d1d4dc7826 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:49.932678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13779ab7-c30c-4a81-9bca-c8d1d4dc7826 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:50.025490 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1347/2682] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:23:49 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:23:50.029506 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d6cb7-faf1-4e9a-839f-b08683d5d693 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:50.040351 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d6cb7-faf1-4e9a-839f-b08683d5d693 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:50.098765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab029af-508b-4efe-9379-93e1a6bd550a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:50.099318 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab029af-508b-4efe-9379-93e1a6bd550a admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:50.099600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab029af-508b-4efe-9379-93e1a6bd550a admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:50.099875 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab029af-508b-4efe-9379-93e1a6bd550a admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:50.100589 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab029af-508b-4efe-9379-93e1a6bd550a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:50.102522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab029af-508b-4efe-9379-93e1a6bd550a admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:50.219492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1336/2683] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:23:50 2022] POST /identity/v3/users => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:23:50.223401 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13779ab7-c30c-4a81-9bca-c8d1d4dc7826 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:50.233471 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13779ab7-c30c-4a81-9bca-c8d1d4dc7826 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:50.238905 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb058b14-1964-4f03-aea9-5d53d6073474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:50.239540 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb058b14-1964-4f03-aea9-5d53d6073474 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:50.239823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb058b14-1964-4f03-aea9-5d53d6073474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:50.240099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb058b14-1964-4f03-aea9-5d53d6073474 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:50.240833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb058b14-1964-4f03-aea9-5d53d6073474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:50.242451 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb058b14-1964-4f03-aea9-5d53d6073474 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:50.253608 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1348/2684] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:23:50.257677 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab029af-508b-4efe-9379-93e1a6bd550a admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:50.267895 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab029af-508b-4efe-9379-93e1a6bd550a admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:50.272494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e221f5-e1c1-4d78-a096-5672dccbc5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:50.273359 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e221f5-e1c1-4d78-a096-5672dccbc5d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:50.273638 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e221f5-e1c1-4d78-a096-5672dccbc5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:50.273913 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e221f5-e1c1-4d78-a096-5672dccbc5d9 admin admin] PATH_INFO: `/v3/projects/645003d2f6014f62884f7d1f4c0afd28/users/642c781c8b3a44868970c517f9cae88e/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:50.289380 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e221f5-e1c1-4d78-a096-5672dccbc5d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=645003d2f6014f62884f7d1f4c0afd28, user_id=642c781c8b3a44868970c517f9cae88e, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:50.292005 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e221f5-e1c1-4d78-a096-5672dccbc5d9 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:50.351103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1337/2685] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:23:50 2022] PUT /identity/v3/projects/645003d2f6014f62884f7d1f4c0afd28/users/642c781c8b3a44868970c517f9cae88e/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:23:50.355067 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb058b14-1964-4f03-aea9-5d53d6073474 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:50.365500 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb058b14-1964-4f03-aea9-5d53d6073474 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:50.422130 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc279e9-83ce-4888-9d68-e2cf8c8463bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:50.422800 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc279e9-83ce-4888-9d68-e2cf8c8463bf admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:50.423084 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc279e9-83ce-4888-9d68-e2cf8c8463bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:50.423911 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc279e9-83ce-4888-9d68-e2cf8c8463bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:50.424649 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc279e9-83ce-4888-9d68-e2cf8c8463bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:50.426271 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc279e9-83ce-4888-9d68-e2cf8c8463bf admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:50.436021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1349/2686] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:23:50.440602 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e221f5-e1c1-4d78-a096-5672dccbc5d9 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:50.450616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e221f5-e1c1-4d78-a096-5672dccbc5d9 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:50.456022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123ecf2-a4f8-408a-b91b-ba0b0f53653f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:50.456889 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123ecf2-a4f8-408a-b91b-ba0b0f53653f admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:50.457175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123ecf2-a4f8-408a-b91b-ba0b0f53653f admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:50.457452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123ecf2-a4f8-408a-b91b-ba0b0f53653f admin admin] PATH_INFO: `/v3/projects/645003d2f6014f62884f7d1f4c0afd28/users/642c781c8b3a44868970c517f9cae88e/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:50.459440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123ecf2-a4f8-408a-b91b-ba0b0f53653f admin admin] RBAC: Authorizing `identity:create_grant(project_id=645003d2f6014f62884f7d1f4c0afd28, user_id=642c781c8b3a44868970c517f9cae88e, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:50.462054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123ecf2-a4f8-408a-b91b-ba0b0f53653f admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:50.472527 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1338/2687] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:23:50 2022] PUT /identity/v3/projects/645003d2f6014f62884f7d1f4c0afd28/users/642c781c8b3a44868970c517f9cae88e/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:23:50.482329 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55912077-570f-4ca0-a5f4-ab38de3781cb None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:50.482626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55912077-570f-4ca0-a5f4-ab38de3781cb None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:50.482903 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55912077-570f-4ca0-a5f4-ab38de3781cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:50.508490 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-55912077-570f-4ca0-a5f4-ab38de3781cb None None] MFA Rules not processed for user `642c781c8b3a44868970c517f9cae88e`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:23:50.509675 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-55912077-570f-4ca0-a5f4-ab38de3781cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:50.573890 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1350/2688] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:23:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:23:50.591187 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123ecf2-a4f8-408a-b91b-ba0b0f53653f admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:50.591187 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0123ecf2-a4f8-408a-b91b-ba0b0f53653f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:23:50.644245 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123ecf2-a4f8-408a-b91b-ba0b0f53653f admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:50.647911 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa7124-88f1-4784-b64e-fbc976bf4c40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:50.648547 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa7124-88f1-4784-b64e-fbc976bf4c40 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:50.648832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa7124-88f1-4784-b64e-fbc976bf4c40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:50.649110 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa7124-88f1-4784-b64e-fbc976bf4c40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:50.704153 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa7124-88f1-4784-b64e-fbc976bf4c40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:50.705917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa7124-88f1-4784-b64e-fbc976bf4c40 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:50.714679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1339/2689] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:23:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:23:55.719950 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55912077-570f-4ca0-a5f4-ab38de3781cb None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:23:55.730426 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55912077-570f-4ca0-a5f4-ab38de3781cb None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:23:55.735038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1aa503-4f6a-4e08-8013-1931b144019b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:23:55.736142 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1aa503-4f6a-4e08-8013-1931b144019b service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:23:55.736644 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1aa503-4f6a-4e08-8013-1931b144019b service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:23:55.737145 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1aa503-4f6a-4e08-8013-1931b144019b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:23:55.748238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1aa503-4f6a-4e08-8013-1931b144019b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:23:55.749661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1aa503-4f6a-4e08-8013-1931b144019b service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:23:55.759903 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1351/2690] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:23:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:24:05.627177 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa7124-88f1-4784-b64e-fbc976bf4c40 service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:05.688210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa7124-88f1-4784-b64e-fbc976bf4c40 service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:05.693300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7c973-e419-4df6-af10-fc16db76a75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:05.694193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7c973-e419-4df6-af10-fc16db76a75e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:05.694480 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7c973-e419-4df6-af10-fc16db76a75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:05.694761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7c973-e419-4df6-af10-fc16db76a75e admin admin] PATH_INFO: `/v3/users/c5f9c320e64844d7bae7e9a0fe96907f` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:05.695973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7c973-e419-4df6-af10-fc16db76a75e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5f9c320e64844d7bae7e9a0fe96907f)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:05.697892 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7c973-e419-4df6-af10-fc16db76a75e admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:05.895616 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-bfc7c973-e419-4df6-af10-fc16db76a75e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5f9c320e64844d7bae7e9a0fe96907f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:05.896264 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-bfc7c973-e419-4df6-af10-fc16db76a75e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5f9c320e64844d7bae7e9a0fe96907f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:05.907737 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-bfc7c973-e419-4df6-af10-fc16db76a75e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5f9c320e64844d7bae7e9a0fe96907f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:05.910862 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-bfc7c973-e419-4df6-af10-fc16db76a75e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5f9c320e64844d7bae7e9a0fe96907f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:05.915215 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-bfc7c973-e419-4df6-af10-fc16db76a75e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5f9c320e64844d7bae7e9a0fe96907f'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:05.991294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1340/2691] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:24:05 2022] DELETE /identity/v3/users/c5f9c320e64844d7bae7e9a0fe96907f => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:24:05.995664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75478991-5024-4a4b-9e21-8cced3186017 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:05.996137 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75478991-5024-4a4b-9e21-8cced3186017 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:05.996251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75478991-5024-4a4b-9e21-8cced3186017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:06.013879 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-75478991-5024-4a4b-9e21-8cced3186017 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:06.015231 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-75478991-5024-4a4b-9e21-8cced3186017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:06.071327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1352/2692] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:24:06.081137 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7c973-e419-4df6-af10-fc16db76a75e admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:06.082562 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-bfc7c973-e419-4df6-af10-fc16db76a75e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:06.139442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7c973-e419-4df6-af10-fc16db76a75e admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:06.143130 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab0df0-6e6b-4d1a-80b6-56a420f098c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:06.143785 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab0df0-6e6b-4d1a-80b6-56a420f098c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:06.144068 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab0df0-6e6b-4d1a-80b6-56a420f098c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:06.144342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab0df0-6e6b-4d1a-80b6-56a420f098c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:06.153254 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab0df0-6e6b-4d1a-80b6-56a420f098c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:06.154717 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab0df0-6e6b-4d1a-80b6-56a420f098c8 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:06.164490 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1341/2693] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:24:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:24:06.538952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75478991-5024-4a4b-9e21-8cced3186017 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:06.540457 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-75478991-5024-4a4b-9e21-8cced3186017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:06.552577 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75478991-5024-4a4b-9e21-8cced3186017 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:06.557163 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceae58d-df55-4747-907f-2ffdb4143c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:06.558032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceae58d-df55-4747-907f-2ffdb4143c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:06.558420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceae58d-df55-4747-907f-2ffdb4143c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:06.558697 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceae58d-df55-4747-907f-2ffdb4143c54 admin admin] PATH_INFO: `/v3/projects/c66479bf2d944babb976591e7ddf569c` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:06.559925 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceae58d-df55-4747-907f-2ffdb4143c54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c66479bf2d944babb976591e7ddf569c)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:06.561830 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceae58d-df55-4747-907f-2ffdb4143c54 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:06.668520 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-bceae58d-df55-4747-907f-2ffdb4143c54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c66479bf2d944babb976591e7ddf569c', 'request_id': 'req-bceae58d-df55-4747-907f-2ffdb4143c54'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:06.754478 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-bceae58d-df55-4747-907f-2ffdb4143c54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c66479bf2d944babb976591e7ddf569c', 'request_id': 'req-bceae58d-df55-4747-907f-2ffdb4143c54'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:06.760416 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-bceae58d-df55-4747-907f-2ffdb4143c54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c66479bf2d944babb976591e7ddf569c', 'request_id': 'req-bceae58d-df55-4747-907f-2ffdb4143c54'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:06.774057 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-bceae58d-df55-4747-907f-2ffdb4143c54 admin admin] The token cache is being invalidate because project c66479bf2d944babb976591e7ddf569c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:24:06.774375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-bceae58d-df55-4747-907f-2ffdb4143c54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:06.775001 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-bceae58d-df55-4747-907f-2ffdb4143c54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:06.776570 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1353/2694] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:24:06 2022] DELETE /identity/v3/projects/c66479bf2d944babb976591e7ddf569c => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:24:06.780649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab0df0-6e6b-4d1a-80b6-56a420f098c8 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:06.791017 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab0df0-6e6b-4d1a-80b6-56a420f098c8 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:06.849051 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e0801-8c96-4634-a181-1e59b612ac0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:06.849922 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e0801-8c96-4634-a181-1e59b612ac0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:06.850237 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e0801-8c96-4634-a181-1e59b612ac0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:06.850514 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e0801-8c96-4634-a181-1e59b612ac0b admin admin] PATH_INFO: `/v3/users/642c781c8b3a44868970c517f9cae88e` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:06.851832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e0801-8c96-4634-a181-1e59b612ac0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=642c781c8b3a44868970c517f9cae88e)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:06.853721 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e0801-8c96-4634-a181-1e59b612ac0b admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:07.032348 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-cc0e0801-8c96-4634-a181-1e59b612ac0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '642c781c8b3a44868970c517f9cae88e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:07.032991 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-cc0e0801-8c96-4634-a181-1e59b612ac0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '642c781c8b3a44868970c517f9cae88e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:07.044899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-cc0e0801-8c96-4634-a181-1e59b612ac0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '642c781c8b3a44868970c517f9cae88e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:07.045179 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-cc0e0801-8c96-4634-a181-1e59b612ac0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '642c781c8b3a44868970c517f9cae88e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:07.051897 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-cc0e0801-8c96-4634-a181-1e59b612ac0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '642c781c8b3a44868970c517f9cae88e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:07.142736 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1342/2695] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:24:06 2022] DELETE /identity/v3/users/642c781c8b3a44868970c517f9cae88e => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:24:07.546760 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceae58d-df55-4747-907f-2ffdb4143c54 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:07.548294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-bceae58d-df55-4747-907f-2ffdb4143c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:07.561798 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceae58d-df55-4747-907f-2ffdb4143c54 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:07.625818 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5dc809-fbaf-422b-b7a0-ca008e767ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:07.626636 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5dc809-fbaf-422b-b7a0-ca008e767ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:07.626923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5dc809-fbaf-422b-b7a0-ca008e767ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:07.627203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5dc809-fbaf-422b-b7a0-ca008e767ccb admin admin] PATH_INFO: `/v3/projects/645003d2f6014f62884f7d1f4c0afd28` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:07.628420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5dc809-fbaf-422b-b7a0-ca008e767ccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=645003d2f6014f62884f7d1f4c0afd28)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:07.630355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5dc809-fbaf-422b-b7a0-ca008e767ccb admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:07.719419 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-fb5dc809-fbaf-422b-b7a0-ca008e767ccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '645003d2f6014f62884f7d1f4c0afd28', 'request_id': 'req-fb5dc809-fbaf-422b-b7a0-ca008e767ccb'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:07.782500 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-fb5dc809-fbaf-422b-b7a0-ca008e767ccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '645003d2f6014f62884f7d1f4c0afd28', 'request_id': 'req-fb5dc809-fbaf-422b-b7a0-ca008e767ccb'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:07.789220 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-fb5dc809-fbaf-422b-b7a0-ca008e767ccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '645003d2f6014f62884f7d1f4c0afd28', 'request_id': 'req-fb5dc809-fbaf-422b-b7a0-ca008e767ccb'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:07.800743 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-fb5dc809-fbaf-422b-b7a0-ca008e767ccb admin admin] The token cache is being invalidate because project 645003d2f6014f62884f7d1f4c0afd28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:24:07.801030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-fb5dc809-fbaf-422b-b7a0-ca008e767ccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:07.801656 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-fb5dc809-fbaf-422b-b7a0-ca008e767ccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:07.803235 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1354/2696] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:24:07 2022] DELETE /identity/v3/projects/645003d2f6014f62884f7d1f4c0afd28 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:24:07.809293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785296a-9752-4572-9063-a3d28acf2d8d None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:07.809781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785296a-9752-4572-9063-a3d28acf2d8d None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:07.809898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785296a-9752-4572-9063-a3d28acf2d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:07.827192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-3785296a-9752-4572-9063-a3d28acf2d8d None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:07.828373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-3785296a-9752-4572-9063-a3d28acf2d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:07.888426 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1343/2697] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:24:07.896803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c00824-b1fe-44a9-a589-810bd001c521 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:07.897123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c00824-b1fe-44a9-a589-810bd001c521 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:07.897403 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c00824-b1fe-44a9-a589-810bd001c521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:07.915933 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-22c00824-b1fe-44a9-a589-810bd001c521 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:07.917138 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-22c00824-b1fe-44a9-a589-810bd001c521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:07.925769 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1355/2698] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:24:07.929960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785296a-9752-4572-9063-a3d28acf2d8d None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:07.939797 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785296a-9752-4572-9063-a3d28acf2d8d None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:07.944291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b498088-d0e6-455d-b887-6560b108b7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:07.944947 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b498088-d0e6-455d-b887-6560b108b7bd admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:07.945228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b498088-d0e6-455d-b887-6560b108b7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:07.945502 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b498088-d0e6-455d-b887-6560b108b7bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:07.946265 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b498088-d0e6-455d-b887-6560b108b7bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:24:07.946574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b498088-d0e6-455d-b887-6560b108b7bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:07.948162 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b498088-d0e6-455d-b887-6560b108b7bd admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:07.960377 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1344/2699] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 20:24:07.964300 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c00824-b1fe-44a9-a589-810bd001c521 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:07.973753 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c00824-b1fe-44a9-a589-810bd001c521 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:07.978450 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72933bd9-b0e0-4a1a-ae8a-ed7635a2afde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:07.979111 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72933bd9-b0e0-4a1a-ae8a-ed7635a2afde admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:07.979391 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72933bd9-b0e0-4a1a-ae8a-ed7635a2afde admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:07.979665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72933bd9-b0e0-4a1a-ae8a-ed7635a2afde admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:07.980377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72933bd9-b0e0-4a1a-ae8a-ed7635a2afde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:07.982282 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72933bd9-b0e0-4a1a-ae8a-ed7635a2afde admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:08.069424 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1356/2700] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:24:07 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:24:08.073636 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b498088-d0e6-455d-b887-6560b108b7bd admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:08.084255 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b498088-d0e6-455d-b887-6560b108b7bd admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:08.138199 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d25201-4cb1-4c1f-b27d-04f919d367a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:08.138920 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d25201-4cb1-4c1f-b27d-04f919d367a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:08.139155 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d25201-4cb1-4c1f-b27d-04f919d367a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:08.139433 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d25201-4cb1-4c1f-b27d-04f919d367a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:08.140172 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d25201-4cb1-4c1f-b27d-04f919d367a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:08.142072 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d25201-4cb1-4c1f-b27d-04f919d367a3 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:08.251150 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1345/2701] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:24:08 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:24:08.255376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72933bd9-b0e0-4a1a-ae8a-ed7635a2afde admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:08.265503 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72933bd9-b0e0-4a1a-ae8a-ed7635a2afde admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:08.270689 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b8f61-74b1-42d6-8b51-6396949a4f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:08.271346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b8f61-74b1-42d6-8b51-6396949a4f1b admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:08.271635 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b8f61-74b1-42d6-8b51-6396949a4f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:08.271912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b8f61-74b1-42d6-8b51-6396949a4f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:08.272671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b8f61-74b1-42d6-8b51-6396949a4f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:08.274286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b8f61-74b1-42d6-8b51-6396949a4f1b admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:08.284354 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1357/2702] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:24:08.288795 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d25201-4cb1-4c1f-b27d-04f919d367a3 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:08.298640 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d25201-4cb1-4c1f-b27d-04f919d367a3 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:08.303242 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152dce72-0a26-4978-84c0-646772f51903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:08.303887 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152dce72-0a26-4978-84c0-646772f51903 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:08.304172 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152dce72-0a26-4978-84c0-646772f51903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:08.304448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152dce72-0a26-4978-84c0-646772f51903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:08.305175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152dce72-0a26-4978-84c0-646772f51903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:08.306880 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152dce72-0a26-4978-84c0-646772f51903 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:08.316504 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1346/2703] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:24:08.320577 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b8f61-74b1-42d6-8b51-6396949a4f1b admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:08.330361 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b8f61-74b1-42d6-8b51-6396949a4f1b admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:08.335597 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e890c6d-a8aa-4cac-9399-f9ee5982c910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:08.336466 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e890c6d-a8aa-4cac-9399-f9ee5982c910 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:08.336782 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e890c6d-a8aa-4cac-9399-f9ee5982c910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:08.337061 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e890c6d-a8aa-4cac-9399-f9ee5982c910 admin admin] PATH_INFO: `/v3/projects/b86c6c438fe548738bc8edd1c73104f8/users/a690945ec4bf48af90a4863c86816be7/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:08.351731 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e890c6d-a8aa-4cac-9399-f9ee5982c910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b86c6c438fe548738bc8edd1c73104f8, user_id=a690945ec4bf48af90a4863c86816be7, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:08.354514 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e890c6d-a8aa-4cac-9399-f9ee5982c910 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:08.465398 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1358/2704] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:24:08 2022] PUT /identity/v3/projects/b86c6c438fe548738bc8edd1c73104f8/users/a690945ec4bf48af90a4863c86816be7/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:24:08.484661 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3556248-b0a6-411a-8851-67b27a4a46ee None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:08.484953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3556248-b0a6-411a-8851-67b27a4a46ee None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:08.485233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3556248-b0a6-411a-8851-67b27a4a46ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:08.512259 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-b3556248-b0a6-411a-8851-67b27a4a46ee None None] MFA Rules not processed for user `a690945ec4bf48af90a4863c86816be7`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:08.513458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b3556248-b0a6-411a-8851-67b27a4a46ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:08.574973 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1347/2705] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:24:08.586387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e890c6d-a8aa-4cac-9399-f9ee5982c910 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:08.588224 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4e890c6d-a8aa-4cac-9399-f9ee5982c910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:08.643114 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e890c6d-a8aa-4cac-9399-f9ee5982c910 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:08.647409 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056ce05-84f1-499d-8ed9-7232bf8d2187 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:08.648044 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056ce05-84f1-499d-8ed9-7232bf8d2187 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:08.648327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056ce05-84f1-499d-8ed9-7232bf8d2187 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:08.648606 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056ce05-84f1-499d-8ed9-7232bf8d2187 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:08.656715 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056ce05-84f1-499d-8ed9-7232bf8d2187 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:08.658171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056ce05-84f1-499d-8ed9-7232bf8d2187 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:08.666811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1359/2706] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:24:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:24:18.520369 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3556248-b0a6-411a-8851-67b27a4a46ee None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:18.580552 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3556248-b0a6-411a-8851-67b27a4a46ee None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:18.585010 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182f0e8-0ace-4d78-a239-f2e5e15dbea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:18.585868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182f0e8-0ace-4d78-a239-f2e5e15dbea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:18.586174 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182f0e8-0ace-4d78-a239-f2e5e15dbea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:18.586449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182f0e8-0ace-4d78-a239-f2e5e15dbea7 admin admin] PATH_INFO: `/v3/users/a690945ec4bf48af90a4863c86816be7` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:18.587638 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182f0e8-0ace-4d78-a239-f2e5e15dbea7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a690945ec4bf48af90a4863c86816be7)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:18.589536 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182f0e8-0ace-4d78-a239-f2e5e15dbea7 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:18.750991 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3182f0e8-0ace-4d78-a239-f2e5e15dbea7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a690945ec4bf48af90a4863c86816be7'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:18.751645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3182f0e8-0ace-4d78-a239-f2e5e15dbea7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a690945ec4bf48af90a4863c86816be7'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:18.763015 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3182f0e8-0ace-4d78-a239-f2e5e15dbea7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a690945ec4bf48af90a4863c86816be7'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:18.763645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3182f0e8-0ace-4d78-a239-f2e5e15dbea7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a690945ec4bf48af90a4863c86816be7'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:18.770246 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-3182f0e8-0ace-4d78-a239-f2e5e15dbea7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a690945ec4bf48af90a4863c86816be7'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:18.854033 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1348/2707] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:24:18 2022] DELETE /identity/v3/users/a690945ec4bf48af90a4863c86816be7 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:24:18.858552 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4b79b-bda7-4011-98f5-0fc151551adb None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:18.859021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4b79b-bda7-4011-98f5-0fc151551adb None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:18.859123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4b79b-bda7-4011-98f5-0fc151551adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:18.876758 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-caf4b79b-bda7-4011-98f5-0fc151551adb None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:18.877949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-caf4b79b-bda7-4011-98f5-0fc151551adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:18.939051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1360/2708] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:24:18.977598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182f0e8-0ace-4d78-a239-f2e5e15dbea7 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:18.977598 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-3182f0e8-0ace-4d78-a239-f2e5e15dbea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:19.018736 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182f0e8-0ace-4d78-a239-f2e5e15dbea7 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:19.022440 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c24e55-dcd6-49d6-bb9c-0b9183c897c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:19.023104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c24e55-dcd6-49d6-bb9c-0b9183c897c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:19.023388 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c24e55-dcd6-49d6-bb9c-0b9183c897c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:19.023666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c24e55-dcd6-49d6-bb9c-0b9183c897c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:19.032835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c24e55-dcd6-49d6-bb9c-0b9183c897c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:19.034327 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c24e55-dcd6-49d6-bb9c-0b9183c897c2 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:19.044445 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1349/2709] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:24:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:24:19.449837 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4b79b-bda7-4011-98f5-0fc151551adb None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:19.451299 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-caf4b79b-bda7-4011-98f5-0fc151551adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:19.464206 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4b79b-bda7-4011-98f5-0fc151551adb None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:19.469051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415708d-90ed-4528-8661-53146af67162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:19.470192 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415708d-90ed-4528-8661-53146af67162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:19.470586 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415708d-90ed-4528-8661-53146af67162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:19.470985 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415708d-90ed-4528-8661-53146af67162 admin admin] PATH_INFO: `/v3/projects/b86c6c438fe548738bc8edd1c73104f8` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:19.472304 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415708d-90ed-4528-8661-53146af67162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b86c6c438fe548738bc8edd1c73104f8)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:19.474340 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415708d-90ed-4528-8661-53146af67162 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:19.595921 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1415708d-90ed-4528-8661-53146af67162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b86c6c438fe548738bc8edd1c73104f8', 'request_id': 'req-1415708d-90ed-4528-8661-53146af67162'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:19.670936 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1415708d-90ed-4528-8661-53146af67162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b86c6c438fe548738bc8edd1c73104f8', 'request_id': 'req-1415708d-90ed-4528-8661-53146af67162'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:19.676509 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1415708d-90ed-4528-8661-53146af67162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b86c6c438fe548738bc8edd1c73104f8', 'request_id': 'req-1415708d-90ed-4528-8661-53146af67162'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:19.687505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1415708d-90ed-4528-8661-53146af67162 admin admin] The token cache is being invalidate because project b86c6c438fe548738bc8edd1c73104f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:24:19.687788 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1415708d-90ed-4528-8661-53146af67162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:19.688500 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1415708d-90ed-4528-8661-53146af67162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:19.690277 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1361/2710] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:24:19 2022] DELETE /identity/v3/projects/b86c6c438fe548738bc8edd1c73104f8 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:24:19.695114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d8470-757d-44d5-8c4f-75dd4d7557d4 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:19.696154 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d8470-757d-44d5-8c4f-75dd4d7557d4 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:19.696287 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d8470-757d-44d5-8c4f-75dd4d7557d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:19.714638 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-a99d8470-757d-44d5-8c4f-75dd4d7557d4 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:19.715819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a99d8470-757d-44d5-8c4f-75dd4d7557d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:19.774345 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1350/2711] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:24:19.782045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828da493-18de-45cc-9348-f3be78617626 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:19.782377 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828da493-18de-45cc-9348-f3be78617626 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:19.782658 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828da493-18de-45cc-9348-f3be78617626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:19.804243 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-828da493-18de-45cc-9348-f3be78617626 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:19.805442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-828da493-18de-45cc-9348-f3be78617626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:19.813307 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1362/2712] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:24:19.817450 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d8470-757d-44d5-8c4f-75dd4d7557d4 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:19.827584 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d8470-757d-44d5-8c4f-75dd4d7557d4 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:19.832312 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1030c-503d-4043-8acc-79d5d0a88237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:19.832957 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1030c-503d-4043-8acc-79d5d0a88237 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:19.833253 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1030c-503d-4043-8acc-79d5d0a88237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:19.833525 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1030c-503d-4043-8acc-79d5d0a88237 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:19.834270 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1030c-503d-4043-8acc-79d5d0a88237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:24:19.834576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1030c-503d-4043-8acc-79d5d0a88237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:19.836734 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1030c-503d-4043-8acc-79d5d0a88237 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:19.847864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1351/2713] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 20:24:19.851925 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828da493-18de-45cc-9348-f3be78617626 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:19.860815 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828da493-18de-45cc-9348-f3be78617626 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:19.865380 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d060439-9193-4782-933f-18f65a857252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:19.866013 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d060439-9193-4782-933f-18f65a857252 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:19.866326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d060439-9193-4782-933f-18f65a857252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:19.866602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d060439-9193-4782-933f-18f65a857252 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:19.867823 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d060439-9193-4782-933f-18f65a857252 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:19.869694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d060439-9193-4782-933f-18f65a857252 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:19.950007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1363/2714] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:24:19 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:24:19.953953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1030c-503d-4043-8acc-79d5d0a88237 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:19.964020 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1030c-503d-4043-8acc-79d5d0a88237 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:20.018933 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676d6a5-c6e9-42d7-991c-7bc4e00dd604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:20.019590 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676d6a5-c6e9-42d7-991c-7bc4e00dd604 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:20.019872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676d6a5-c6e9-42d7-991c-7bc4e00dd604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:20.020145 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676d6a5-c6e9-42d7-991c-7bc4e00dd604 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:20.020838 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676d6a5-c6e9-42d7-991c-7bc4e00dd604 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:20.022772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676d6a5-c6e9-42d7-991c-7bc4e00dd604 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:20.094123 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1352/2715] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:24:19 2022] POST /identity/v3/users => generated 288 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:24:20.098120 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d060439-9193-4782-933f-18f65a857252 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:20.108269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d060439-9193-4782-933f-18f65a857252 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:20.112961 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2fef0-7f36-4875-b895-fb6c48486aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:20.113611 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2fef0-7f36-4875-b895-fb6c48486aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:20.113899 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2fef0-7f36-4875-b895-fb6c48486aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:20.114216 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2fef0-7f36-4875-b895-fb6c48486aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:20.114948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2fef0-7f36-4875-b895-fb6c48486aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:20.116537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2fef0-7f36-4875-b895-fb6c48486aa6 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:20.127196 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1364/2716] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:24:20.131101 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676d6a5-c6e9-42d7-991c-7bc4e00dd604 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:20.141709 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676d6a5-c6e9-42d7-991c-7bc4e00dd604 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:20.146373 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64664b-6e42-4e7b-b6bf-2922c66b8b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:20.147022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64664b-6e42-4e7b-b6bf-2922c66b8b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:20.147311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64664b-6e42-4e7b-b6bf-2922c66b8b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:20.147589 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64664b-6e42-4e7b-b6bf-2922c66b8b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:20.148346 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64664b-6e42-4e7b-b6bf-2922c66b8b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:20.149930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64664b-6e42-4e7b-b6bf-2922c66b8b3a admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:20.159683 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1353/2717] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:24:20.163790 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2fef0-7f36-4875-b895-fb6c48486aa6 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:20.173443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2fef0-7f36-4875-b895-fb6c48486aa6 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:20.178005 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622d4d3-5e5e-4ab8-8252-7757d7e78af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:20.178914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622d4d3-5e5e-4ab8-8252-7757d7e78af0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:20.179182 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622d4d3-5e5e-4ab8-8252-7757d7e78af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:20.179459 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622d4d3-5e5e-4ab8-8252-7757d7e78af0 admin admin] PATH_INFO: `/v3/projects/48701857a09c4345b70e989e5529865d/users/c5fb7ef5c8d84ddebf1e11b813cacf99/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:20.193935 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622d4d3-5e5e-4ab8-8252-7757d7e78af0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48701857a09c4345b70e989e5529865d, user_id=c5fb7ef5c8d84ddebf1e11b813cacf99, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:20.197232 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622d4d3-5e5e-4ab8-8252-7757d7e78af0 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:20.266063 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1365/2718] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:24:20 2022] PUT /identity/v3/projects/48701857a09c4345b70e989e5529865d/users/c5fb7ef5c8d84ddebf1e11b813cacf99/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:24:20.280718 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e42bd-84f5-4cc1-8c14-b956193dd193 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:20.281020 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e42bd-84f5-4cc1-8c14-b956193dd193 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:20.281301 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e42bd-84f5-4cc1-8c14-b956193dd193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:20.306995 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-8e9e42bd-84f5-4cc1-8c14-b956193dd193 None None] MFA Rules not processed for user `c5fb7ef5c8d84ddebf1e11b813cacf99`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:20.308208 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8e9e42bd-84f5-4cc1-8c14-b956193dd193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:20.372210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1354/2719] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:24:20.383584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622d4d3-5e5e-4ab8-8252-7757d7e78af0 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:20.385290 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b622d4d3-5e5e-4ab8-8252-7757d7e78af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:20.439424 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622d4d3-5e5e-4ab8-8252-7757d7e78af0 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:20.443088 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c60dfd-7336-4709-8215-f9fabbaa3558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:20.443726 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c60dfd-7336-4709-8215-f9fabbaa3558 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:20.444011 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c60dfd-7336-4709-8215-f9fabbaa3558 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:20.444288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c60dfd-7336-4709-8215-f9fabbaa3558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:20.452907 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c60dfd-7336-4709-8215-f9fabbaa3558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:20.454352 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c60dfd-7336-4709-8215-f9fabbaa3558 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:20.463646 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1366/2720] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:24:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:24:26.681192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e42bd-84f5-4cc1-8c14-b956193dd193 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:26.745467 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e42bd-84f5-4cc1-8c14-b956193dd193 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:26.750055 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88962190-aaac-4954-aaae-62868249f25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:26.750952 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88962190-aaac-4954-aaae-62868249f25a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:26.751273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88962190-aaac-4954-aaae-62868249f25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:26.751538 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88962190-aaac-4954-aaae-62868249f25a admin admin] PATH_INFO: `/v3/users/c5fb7ef5c8d84ddebf1e11b813cacf99` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:26.752747 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88962190-aaac-4954-aaae-62868249f25a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5fb7ef5c8d84ddebf1e11b813cacf99)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:26.754705 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88962190-aaac-4954-aaae-62868249f25a admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:26.973543 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-88962190-aaac-4954-aaae-62868249f25a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5fb7ef5c8d84ddebf1e11b813cacf99'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:26.974227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-88962190-aaac-4954-aaae-62868249f25a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5fb7ef5c8d84ddebf1e11b813cacf99'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:26.986168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-88962190-aaac-4954-aaae-62868249f25a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5fb7ef5c8d84ddebf1e11b813cacf99'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:26.986793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-88962190-aaac-4954-aaae-62868249f25a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5fb7ef5c8d84ddebf1e11b813cacf99'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:26.994007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-88962190-aaac-4954-aaae-62868249f25a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5fb7ef5c8d84ddebf1e11b813cacf99'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:27.111820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1355/2721] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:24:26 2022] DELETE /identity/v3/users/c5fb7ef5c8d84ddebf1e11b813cacf99 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:24:27.116418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d3dc9-cdec-4627-980e-c514e924a9b9 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:27.116917 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d3dc9-cdec-4627-980e-c514e924a9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:27.117132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d3dc9-cdec-4627-980e-c514e924a9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:27.137446 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-351d3dc9-cdec-4627-980e-c514e924a9b9 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:27.137446 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-351d3dc9-cdec-4627-980e-c514e924a9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:27.193863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1367/2722] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:24:27.208126 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88962190-aaac-4954-aaae-62868249f25a admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:27.208126 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-88962190-aaac-4954-aaae-62868249f25a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:27.266332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88962190-aaac-4954-aaae-62868249f25a admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:27.269833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca42ec-86ce-44fa-8f27-fad39510ac56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:27.270498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca42ec-86ce-44fa-8f27-fad39510ac56 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:27.270777 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca42ec-86ce-44fa-8f27-fad39510ac56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:27.271047 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca42ec-86ce-44fa-8f27-fad39510ac56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:27.280008 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca42ec-86ce-44fa-8f27-fad39510ac56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:27.281568 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca42ec-86ce-44fa-8f27-fad39510ac56 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:27.291983 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1356/2723] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:24:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:24:27.704314 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d3dc9-cdec-4627-980e-c514e924a9b9 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:27.705967 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-351d3dc9-cdec-4627-980e-c514e924a9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:27.719253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d3dc9-cdec-4627-980e-c514e924a9b9 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:27.724283 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceff4ef-bfaf-435b-82dc-d3e271e7f136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:27.729065 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceff4ef-bfaf-435b-82dc-d3e271e7f136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:27.729065 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceff4ef-bfaf-435b-82dc-d3e271e7f136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:27.729065 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceff4ef-bfaf-435b-82dc-d3e271e7f136 admin admin] PATH_INFO: `/v3/projects/48701857a09c4345b70e989e5529865d` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:27.729065 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceff4ef-bfaf-435b-82dc-d3e271e7f136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48701857a09c4345b70e989e5529865d)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:27.729465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceff4ef-bfaf-435b-82dc-d3e271e7f136 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:27.835293 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2ceff4ef-bfaf-435b-82dc-d3e271e7f136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48701857a09c4345b70e989e5529865d', 'request_id': 'req-2ceff4ef-bfaf-435b-82dc-d3e271e7f136'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:27.927817 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2ceff4ef-bfaf-435b-82dc-d3e271e7f136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48701857a09c4345b70e989e5529865d', 'request_id': 'req-2ceff4ef-bfaf-435b-82dc-d3e271e7f136'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:27.933677 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2ceff4ef-bfaf-435b-82dc-d3e271e7f136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48701857a09c4345b70e989e5529865d', 'request_id': 'req-2ceff4ef-bfaf-435b-82dc-d3e271e7f136'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:27.944602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2ceff4ef-bfaf-435b-82dc-d3e271e7f136 admin admin] The token cache is being invalidate because project 48701857a09c4345b70e989e5529865d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:24:27.944892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2ceff4ef-bfaf-435b-82dc-d3e271e7f136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:27.946017 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2ceff4ef-bfaf-435b-82dc-d3e271e7f136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:27.947584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1368/2724] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:24:27 2022] DELETE /identity/v3/projects/48701857a09c4345b70e989e5529865d => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:24:27.952549 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a480bfe-fff8-44b7-b96d-c1c49681f497 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:27.953004 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a480bfe-fff8-44b7-b96d-c1c49681f497 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:27.953286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a480bfe-fff8-44b7-b96d-c1c49681f497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:27.971472 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-4a480bfe-fff8-44b7-b96d-c1c49681f497 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:27.972586 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-4a480bfe-fff8-44b7-b96d-c1c49681f497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:28.030781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1357/2725] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:24:28.038649 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7898331-0192-425b-8dc4-e05cee694911 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:28.038943 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7898331-0192-425b-8dc4-e05cee694911 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:28.039290 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7898331-0192-425b-8dc4-e05cee694911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:28.057022 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-f7898331-0192-425b-8dc4-e05cee694911 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:28.058261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-f7898331-0192-425b-8dc4-e05cee694911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:28.066056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1369/2726] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:24:28.070828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a480bfe-fff8-44b7-b96d-c1c49681f497 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:28.080495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a480bfe-fff8-44b7-b96d-c1c49681f497 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:28.085037 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e43a56-cf1b-45c4-a020-571095790dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:28.085680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e43a56-cf1b-45c4-a020-571095790dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:28.085962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e43a56-cf1b-45c4-a020-571095790dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:28.086841 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e43a56-cf1b-45c4-a020-571095790dc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:28.087584 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e43a56-cf1b-45c4-a020-571095790dc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:24:28.087885 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e43a56-cf1b-45c4-a020-571095790dc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:28.089510 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e43a56-cf1b-45c4-a020-571095790dc8 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:28.105523 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1358/2727] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 20:24:28.108357 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7898331-0192-425b-8dc4-e05cee694911 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:28.118687 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7898331-0192-425b-8dc4-e05cee694911 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:28.123856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29ed05-c1c5-4454-b8a8-46c5cd90a02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:28.124546 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29ed05-c1c5-4454-b8a8-46c5cd90a02a admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:28.124834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29ed05-c1c5-4454-b8a8-46c5cd90a02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:28.125117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29ed05-c1c5-4454-b8a8-46c5cd90a02a admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:28.125833 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29ed05-c1c5-4454-b8a8-46c5cd90a02a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:28.127820 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29ed05-c1c5-4454-b8a8-46c5cd90a02a admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:28.234469 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e43a56-cf1b-45c4-a020-571095790dc8 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:28.236206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f5e43a56-cf1b-45c4-a020-571095790dc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:28.238600 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1370/2728] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:24:28 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:24:28.243142 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29ed05-c1c5-4454-b8a8-46c5cd90a02a admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:28.297301 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e43a56-cf1b-45c4-a020-571095790dc8 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:28.312430 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29ed05-c1c5-4454-b8a8-46c5cd90a02a admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:28.317173 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248222f-07df-45b4-ac69-9c0d0517b2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:28.317842 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248222f-07df-45b4-ac69-9c0d0517b2c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:28.318164 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248222f-07df-45b4-ac69-9c0d0517b2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:28.318564 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248222f-07df-45b4-ac69-9c0d0517b2c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:28.319276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248222f-07df-45b4-ac69-9c0d0517b2c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:28.321827 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248222f-07df-45b4-ac69-9c0d0517b2c4 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:28.345593 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4be44c-9773-4e66-abc4-82cb029c0677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:28.346276 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4be44c-9773-4e66-abc4-82cb029c0677 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:28.346561 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4be44c-9773-4e66-abc4-82cb029c0677 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:28.346837 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4be44c-9773-4e66-abc4-82cb029c0677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:28.348613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-ff4be44c-9773-4e66-abc4-82cb029c0677 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:28.360701 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4be44c-9773-4e66-abc4-82cb029c0677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:28.362219 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4be44c-9773-4e66-abc4-82cb029c0677 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:28.372504 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1359/2729] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:24:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:24:28.379587 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1371/2730] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:24:28 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:24:28.384200 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4be44c-9773-4e66-abc4-82cb029c0677 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:28.394198 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4be44c-9773-4e66-abc4-82cb029c0677 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:28.398759 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e09dc9-37e7-4eaa-9e14-a322a31d6270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:28.399406 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e09dc9-37e7-4eaa-9e14-a322a31d6270 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:28.399844 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e09dc9-37e7-4eaa-9e14-a322a31d6270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:28.400118 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e09dc9-37e7-4eaa-9e14-a322a31d6270 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:28.400839 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e09dc9-37e7-4eaa-9e14-a322a31d6270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:28.402456 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e09dc9-37e7-4eaa-9e14-a322a31d6270 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:28.412442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1360/2731] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:24:28.416433 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248222f-07df-45b4-ac69-9c0d0517b2c4 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:28.426802 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248222f-07df-45b4-ac69-9c0d0517b2c4 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:28.432577 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656cf3ed-d998-47a5-84e0-37955651a944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:28.433099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cf3ed-d998-47a5-84e0-37955651a944 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:28.433384 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cf3ed-d998-47a5-84e0-37955651a944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:28.433667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cf3ed-d998-47a5-84e0-37955651a944 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:28.434440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656cf3ed-d998-47a5-84e0-37955651a944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:28.436100 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656cf3ed-d998-47a5-84e0-37955651a944 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:28.449953 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1372/2732] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:24:28.451362 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e09dc9-37e7-4eaa-9e14-a322a31d6270 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:28.461793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e09dc9-37e7-4eaa-9e14-a322a31d6270 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:28.466431 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdef896-6ad1-43c7-8f82-3d85b1a267a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:28.467317 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdef896-6ad1-43c7-8f82-3d85b1a267a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:28.467603 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdef896-6ad1-43c7-8f82-3d85b1a267a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:28.467881 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdef896-6ad1-43c7-8f82-3d85b1a267a1 admin admin] PATH_INFO: `/v3/projects/7aba95b1cda144e4b215c57b8455f2a8/users/3b341bd2d2964d74b87b4f3c2b654da5/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:28.485208 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdef896-6ad1-43c7-8f82-3d85b1a267a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aba95b1cda144e4b215c57b8455f2a8, user_id=3b341bd2d2964d74b87b4f3c2b654da5, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:28.487854 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdef896-6ad1-43c7-8f82-3d85b1a267a1 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:28.580680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1361/2733] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:24:28 2022] PUT /identity/v3/projects/7aba95b1cda144e4b215c57b8455f2a8/users/3b341bd2d2964d74b87b4f3c2b654da5/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:24:28.591736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98041a4d-946f-46c1-af4a-59a4f17f9fc6 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:28.592028 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98041a4d-946f-46c1-af4a-59a4f17f9fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:28.592309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98041a4d-946f-46c1-af4a-59a4f17f9fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:28.632626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-98041a4d-946f-46c1-af4a-59a4f17f9fc6 None None] MFA Rules not processed for user `3b341bd2d2964d74b87b4f3c2b654da5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:28.634429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-98041a4d-946f-46c1-af4a-59a4f17f9fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:28.700921 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1373/2734] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:24:28.713165 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdef896-6ad1-43c7-8f82-3d85b1a267a1 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:28.714909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-acdef896-6ad1-43c7-8f82-3d85b1a267a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:28.770886 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdef896-6ad1-43c7-8f82-3d85b1a267a1 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:28.775021 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125fdea-c9af-4770-9359-913d479f30ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:28.775671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125fdea-c9af-4770-9359-913d479f30ef service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:28.775950 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125fdea-c9af-4770-9359-913d479f30ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:28.776228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125fdea-c9af-4770-9359-913d479f30ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:28.784608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125fdea-c9af-4770-9359-913d479f30ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:28.786032 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125fdea-c9af-4770-9359-913d479f30ef service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:28.794788 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1362/2735] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:24:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:24:31.211812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2523f-d969-4836-8327-2901ac7ef204 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:31.212324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2523f-d969-4836-8327-2901ac7ef204 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:31.212324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2523f-d969-4836-8327-2901ac7ef204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:31.231772 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-74e2523f-d969-4836-8327-2901ac7ef204 None None] MFA Rules not processed for user `77e46909519247119a6bc011b6bcfc7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:31.232990 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-74e2523f-d969-4836-8327-2901ac7ef204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:31.287897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1374/2736] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:24:32.310041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125fdea-c9af-4770-9359-913d479f30ef service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:32.311576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-2125fdea-c9af-4770-9359-913d479f30ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:32.378641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125fdea-c9af-4770-9359-913d479f30ef service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:32.385448 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e5571-da1f-4b7b-8a35-2a52e24e06cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:32.386424 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e5571-da1f-4b7b-8a35-2a52e24e06cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:32.387863 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e5571-da1f-4b7b-8a35-2a52e24e06cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:32.387863 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e5571-da1f-4b7b-8a35-2a52e24e06cb admin admin] PATH_INFO: `/v3/users/8feebf7f09cb4f7d81759f8f1336dba5` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:32.388226 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e5571-da1f-4b7b-8a35-2a52e24e06cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8feebf7f09cb4f7d81759f8f1336dba5)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:32.390224 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e5571-da1f-4b7b-8a35-2a52e24e06cb admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:32.588404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-240e5571-da1f-4b7b-8a35-2a52e24e06cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8feebf7f09cb4f7d81759f8f1336dba5'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:32.589064 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-240e5571-da1f-4b7b-8a35-2a52e24e06cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8feebf7f09cb4f7d81759f8f1336dba5'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:32.600867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-240e5571-da1f-4b7b-8a35-2a52e24e06cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8feebf7f09cb4f7d81759f8f1336dba5'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:32.601659 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-240e5571-da1f-4b7b-8a35-2a52e24e06cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8feebf7f09cb4f7d81759f8f1336dba5'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:32.609225 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-240e5571-da1f-4b7b-8a35-2a52e24e06cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8feebf7f09cb4f7d81759f8f1336dba5'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:32.687002 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1363/2737] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:24:32 2022] DELETE /identity/v3/users/8feebf7f09cb4f7d81759f8f1336dba5 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:24:32.900312 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2523f-d969-4836-8327-2901ac7ef204 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:32.911205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2523f-d969-4836-8327-2901ac7ef204 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:32.966230 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332c5ba-f59f-41a0-b88e-5ad45f98f60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:32.967097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332c5ba-f59f-41a0-b88e-5ad45f98f60a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:32.967383 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332c5ba-f59f-41a0-b88e-5ad45f98f60a admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:32.967661 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332c5ba-f59f-41a0-b88e-5ad45f98f60a admin admin] PATH_INFO: `/v3/users/3b341bd2d2964d74b87b4f3c2b654da5` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:32.968843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332c5ba-f59f-41a0-b88e-5ad45f98f60a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b341bd2d2964d74b87b4f3c2b654da5)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:32.970834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332c5ba-f59f-41a0-b88e-5ad45f98f60a admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:32.986790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e5571-da1f-4b7b-8a35-2a52e24e06cb admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:32.987658 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-240e5571-da1f-4b7b-8a35-2a52e24e06cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:33.001421 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e5571-da1f-4b7b-8a35-2a52e24e06cb admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:33.006074 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d7ac4-af01-4b6a-ae1f-8f1a65d31c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:33.006971 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d7ac4-af01-4b6a-ae1f-8f1a65d31c0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:33.007258 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d7ac4-af01-4b6a-ae1f-8f1a65d31c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:33.007548 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d7ac4-af01-4b6a-ae1f-8f1a65d31c0a admin admin] PATH_INFO: `/v3/projects/d5cecfb8eb5e4b8ea121c2ef8a9c28a2` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:33.009362 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d7ac4-af01-4b6a-ae1f-8f1a65d31c0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5cecfb8eb5e4b8ea121c2ef8a9c28a2)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:33.025232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d7ac4-af01-4b6a-ae1f-8f1a65d31c0a admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:33.169000 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-055d7ac4-af01-4b6a-ae1f-8f1a65d31c0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5cecfb8eb5e4b8ea121c2ef8a9c28a2', 'request_id': 'req-055d7ac4-af01-4b6a-ae1f-8f1a65d31c0a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:33.169214 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2332c5ba-f59f-41a0-b88e-5ad45f98f60a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b341bd2d2964d74b87b4f3c2b654da5'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:33.169767 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2332c5ba-f59f-41a0-b88e-5ad45f98f60a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b341bd2d2964d74b87b4f3c2b654da5'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:33.170425 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2332c5ba-f59f-41a0-b88e-5ad45f98f60a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b341bd2d2964d74b87b4f3c2b654da5'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:33.182526 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2332c5ba-f59f-41a0-b88e-5ad45f98f60a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b341bd2d2964d74b87b4f3c2b654da5'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:33.189293 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-2332c5ba-f59f-41a0-b88e-5ad45f98f60a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b341bd2d2964d74b87b4f3c2b654da5'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:33.236730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-055d7ac4-af01-4b6a-ae1f-8f1a65d31c0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5cecfb8eb5e4b8ea121c2ef8a9c28a2', 'request_id': 'req-055d7ac4-af01-4b6a-ae1f-8f1a65d31c0a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:33.242646 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-055d7ac4-af01-4b6a-ae1f-8f1a65d31c0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5cecfb8eb5e4b8ea121c2ef8a9c28a2', 'request_id': 'req-055d7ac4-af01-4b6a-ae1f-8f1a65d31c0a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:33.253713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-055d7ac4-af01-4b6a-ae1f-8f1a65d31c0a admin admin] The token cache is being invalidate because project d5cecfb8eb5e4b8ea121c2ef8a9c28a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:24:33.253993 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-055d7ac4-af01-4b6a-ae1f-8f1a65d31c0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:33.254641 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-055d7ac4-af01-4b6a-ae1f-8f1a65d31c0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:33.256211 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1364/2738] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:24:32 2022] DELETE /identity/v3/projects/d5cecfb8eb5e4b8ea121c2ef8a9c28a2 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:24:33.261535 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19beb6ad-dcd3-499c-8f21-06992f0051d5 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:33.261824 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19beb6ad-dcd3-499c-8f21-06992f0051d5 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:33.262121 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19beb6ad-dcd3-499c-8f21-06992f0051d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:33.282901 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-19beb6ad-dcd3-499c-8f21-06992f0051d5 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:33.284138 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-19beb6ad-dcd3-499c-8f21-06992f0051d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:33.301691 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1375/2739] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:24:32 2022] DELETE /identity/v3/users/3b341bd2d2964d74b87b4f3c2b654da5 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:24:33.304417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b7ddc-99d0-4694-846a-e12c740a9cf0 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:33.304706 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b7ddc-99d0-4694-846a-e12c740a9cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:33.304979 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b7ddc-99d0-4694-846a-e12c740a9cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:33.322680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-453b7ddc-99d0-4694-846a-e12c740a9cf0 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:33.323938 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-453b7ddc-99d0-4694-846a-e12c740a9cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:33.352332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1365/2740] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:24:33.362004 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5aef8-bccb-48f2-a875-74a767eee05f None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:33.362323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5aef8-bccb-48f2-a875-74a767eee05f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:33.364232 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5aef8-bccb-48f2-a875-74a767eee05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:33.382176 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-f8e5aef8-bccb-48f2-a875-74a767eee05f None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:33.383986 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f8e5aef8-bccb-48f2-a875-74a767eee05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:33.397071 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1376/2741] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:24:33.406526 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b7ddc-99d0-4694-846a-e12c740a9cf0 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:33.407921 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-453b7ddc-99d0-4694-846a-e12c740a9cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:33.456312 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1366/2742] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:24:33.460470 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5aef8-bccb-48f2-a875-74a767eee05f None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:33.466789 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b7ddc-99d0-4694-846a-e12c740a9cf0 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:33.469138 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5aef8-bccb-48f2-a875-74a767eee05f None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:33.470560 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec79b7-17a4-4dd5-bbcc-9cac1dd888ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:33.473254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec79b7-17a4-4dd5-bbcc-9cac1dd888ed service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:33.473254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec79b7-17a4-4dd5-bbcc-9cac1dd888ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:33.473254 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec79b7-17a4-4dd5-bbcc-9cac1dd888ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:33.474519 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7125090-23c3-43fd-a12b-79f4a0348f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:33.475229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7125090-23c3-43fd-a12b-79f4a0348f42 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:33.475452 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7125090-23c3-43fd-a12b-79f4a0348f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:33.475730 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7125090-23c3-43fd-a12b-79f4a0348f42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:33.476459 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7125090-23c3-43fd-a12b-79f4a0348f42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:24:33.476908 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7125090-23c3-43fd-a12b-79f4a0348f42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:33.478593 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7125090-23c3-43fd-a12b-79f4a0348f42 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:33.481403 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec79b7-17a4-4dd5-bbcc-9cac1dd888ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:33.482868 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec79b7-17a4-4dd5-bbcc-9cac1dd888ed service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:33.494213 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1367/2743] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 20:24:33.494408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1377/2744] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:24:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:24:33.495615 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7125090-23c3-43fd-a12b-79f4a0348f42 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:33.506168 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7125090-23c3-43fd-a12b-79f4a0348f42 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:33.511530 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591687b-1d8e-4cd9-a16a-adb5c239cac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:33.512210 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591687b-1d8e-4cd9-a16a-adb5c239cac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:33.512494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591687b-1d8e-4cd9-a16a-adb5c239cac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:33.512771 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591687b-1d8e-4cd9-a16a-adb5c239cac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:33.513457 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591687b-1d8e-4cd9-a16a-adb5c239cac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:33.515394 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591687b-1d8e-4cd9-a16a-adb5c239cac1 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:33.609546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1368/2745] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:24:33 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:24:33.613654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec79b7-17a4-4dd5-bbcc-9cac1dd888ed service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:33.623787 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec79b7-17a4-4dd5-bbcc-9cac1dd888ed service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:33.683726 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52df06-d590-469b-b7f5-5b9f4f238d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:33.684398 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52df06-d590-469b-b7f5-5b9f4f238d6f admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:33.684678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52df06-d590-469b-b7f5-5b9f4f238d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:33.684948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52df06-d590-469b-b7f5-5b9f4f238d6f admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:33.685646 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52df06-d590-469b-b7f5-5b9f4f238d6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:33.687736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52df06-d590-469b-b7f5-5b9f4f238d6f admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:33.824226 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1378/2746] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:24:33 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:24:33.828436 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591687b-1d8e-4cd9-a16a-adb5c239cac1 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:33.839350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591687b-1d8e-4cd9-a16a-adb5c239cac1 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:33.843878 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0ff0e-6002-4da9-bf1d-94d9d81722be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:33.844820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0ff0e-6002-4da9-bf1d-94d9d81722be admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:33.845109 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0ff0e-6002-4da9-bf1d-94d9d81722be admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:33.845393 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0ff0e-6002-4da9-bf1d-94d9d81722be admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:33.846159 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc0ff0e-6002-4da9-bf1d-94d9d81722be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:33.847762 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc0ff0e-6002-4da9-bf1d-94d9d81722be admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:33.861630 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1369/2747] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:24:33.862903 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52df06-d590-469b-b7f5-5b9f4f238d6f admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:33.875509 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52df06-d590-469b-b7f5-5b9f4f238d6f admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:33.880362 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93535cf9-62de-4b6b-b605-74b7a23fb8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:33.881057 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93535cf9-62de-4b6b-b605-74b7a23fb8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:33.881346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93535cf9-62de-4b6b-b605-74b7a23fb8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:33.881654 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93535cf9-62de-4b6b-b605-74b7a23fb8d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:33.882437 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93535cf9-62de-4b6b-b605-74b7a23fb8d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:33.884066 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93535cf9-62de-4b6b-b605-74b7a23fb8d2 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:33.894101 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1379/2748] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:24:33.898218 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0ff0e-6002-4da9-bf1d-94d9d81722be admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:33.908460 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0ff0e-6002-4da9-bf1d-94d9d81722be admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:33.912994 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30af4c-94a4-4380-88c7-adf45b277c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:33.913880 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30af4c-94a4-4380-88c7-adf45b277c22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:33.914699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30af4c-94a4-4380-88c7-adf45b277c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:33.914972 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30af4c-94a4-4380-88c7-adf45b277c22 admin admin] PATH_INFO: `/v3/projects/1d1c68d15d634e81acba3ca215f5548c/users/04ce1bdd6a234177ba7745d19d6fd950/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:33.929066 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30af4c-94a4-4380-88c7-adf45b277c22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d1c68d15d634e81acba3ca215f5548c, user_id=04ce1bdd6a234177ba7745d19d6fd950, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:33.932351 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30af4c-94a4-4380-88c7-adf45b277c22 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:33.962402 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93535cf9-62de-4b6b-b605-74b7a23fb8d2 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:33.964229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-93535cf9-62de-4b6b-b605-74b7a23fb8d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:33.979223 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93535cf9-62de-4b6b-b605-74b7a23fb8d2 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:33.984392 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28892f15-3e63-42e7-bd24-58d0245d2824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:33.985415 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28892f15-3e63-42e7-bd24-58d0245d2824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:33.988253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28892f15-3e63-42e7-bd24-58d0245d2824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:33.988253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28892f15-3e63-42e7-bd24-58d0245d2824 admin admin] PATH_INFO: `/v3/projects/7aba95b1cda144e4b215c57b8455f2a8` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:33.988253 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28892f15-3e63-42e7-bd24-58d0245d2824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aba95b1cda144e4b215c57b8455f2a8)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:33.989445 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28892f15-3e63-42e7-bd24-58d0245d2824 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:34.012874 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1370/2749] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:24:33 2022] PUT /identity/v3/projects/1d1c68d15d634e81acba3ca215f5548c/users/04ce1bdd6a234177ba7745d19d6fd950/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:24:34.017671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8e2cd-b0c3-4ca8-bc86-4fdb7918c74c None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:34.017968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8e2cd-b0c3-4ca8-bc86-4fdb7918c74c None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:34.018283 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8e2cd-b0c3-4ca8-bc86-4fdb7918c74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:34.034937 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-fba8e2cd-b0c3-4ca8-bc86-4fdb7918c74c None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:34.036146 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-fba8e2cd-b0c3-4ca8-bc86-4fdb7918c74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:34.093196 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1371/2750] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:24:34.103567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8e2cd-b0c3-4ca8-bc86-4fdb7918c74c None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:34.105324 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-fba8e2cd-b0c3-4ca8-bc86-4fdb7918c74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:34.113542 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-28892f15-3e63-42e7-bd24-58d0245d2824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aba95b1cda144e4b215c57b8455f2a8', 'request_id': 'req-28892f15-3e63-42e7-bd24-58d0245d2824'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:34.162660 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8e2cd-b0c3-4ca8-bc86-4fdb7918c74c None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:34.194970 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-28892f15-3e63-42e7-bd24-58d0245d2824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aba95b1cda144e4b215c57b8455f2a8', 'request_id': 'req-28892f15-3e63-42e7-bd24-58d0245d2824'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:34.201291 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-28892f15-3e63-42e7-bd24-58d0245d2824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aba95b1cda144e4b215c57b8455f2a8', 'request_id': 'req-28892f15-3e63-42e7-bd24-58d0245d2824'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:34.212942 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57799f-5730-46d9-a131-f01b457c5668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:34.213551 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57799f-5730-46d9-a131-f01b457c5668 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:34.213832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57799f-5730-46d9-a131-f01b457c5668 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:34.213962 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-28892f15-3e63-42e7-bd24-58d0245d2824 admin admin] The token cache is being invalidate because project 7aba95b1cda144e4b215c57b8455f2a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:24:34.214125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57799f-5730-46d9-a131-f01b457c5668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:34.214250 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-28892f15-3e63-42e7-bd24-58d0245d2824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:34.214877 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-28892f15-3e63-42e7-bd24-58d0245d2824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:24:34.215929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-be57799f-5730-46d9-a131-f01b457c5668 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:34.216601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1380/2751] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:24:33 2022] DELETE /identity/v3/projects/7aba95b1cda144e4b215c57b8455f2a8 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:24:34.221124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf1e1a-1841-4640-891d-349723232601 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:34.221414 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf1e1a-1841-4640-891d-349723232601 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:34.221694 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf1e1a-1841-4640-891d-349723232601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:34.246281 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-cfbf1e1a-1841-4640-891d-349723232601 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:34.247534 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-cfbf1e1a-1841-4640-891d-349723232601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:34.291378 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57799f-5730-46d9-a131-f01b457c5668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:34.292977 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57799f-5730-46d9-a131-f01b457c5668 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:34.303113 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1372/2752] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:24:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:24:34.310556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1381/2753] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:24:34.319620 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4026b-77d4-45b4-871d-8c883efded8b None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:34.319904 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4026b-77d4-45b4-871d-8c883efded8b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:34.320181 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4026b-77d4-45b4-871d-8c883efded8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:34.343004 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-83f4026b-77d4-45b4-871d-8c883efded8b None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:34.344233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-83f4026b-77d4-45b4-871d-8c883efded8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:34.353751 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1373/2754] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:24:34.358265 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf1e1a-1841-4640-891d-349723232601 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:34.367985 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf1e1a-1841-4640-891d-349723232601 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:34.372707 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542938b-3d6b-450d-81fe-5b4f33551428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:34.377560 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542938b-3d6b-450d-81fe-5b4f33551428 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:34.377560 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542938b-3d6b-450d-81fe-5b4f33551428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:34.377560 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542938b-3d6b-450d-81fe-5b4f33551428 admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:34.377560 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542938b-3d6b-450d-81fe-5b4f33551428 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:24:34.377560 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542938b-3d6b-450d-81fe-5b4f33551428 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:34.377560 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542938b-3d6b-450d-81fe-5b4f33551428 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:34.388378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1382/2755] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:24:34.392897 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4026b-77d4-45b4-871d-8c883efded8b None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:34.402953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4026b-77d4-45b4-871d-8c883efded8b None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:34.408303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e07cac-f6bc-4cf7-9142-b17f6dea0bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:34.408968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e07cac-f6bc-4cf7-9142-b17f6dea0bae admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:34.409254 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e07cac-f6bc-4cf7-9142-b17f6dea0bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:34.409531 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e07cac-f6bc-4cf7-9142-b17f6dea0bae admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:34.410281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e07cac-f6bc-4cf7-9142-b17f6dea0bae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:34.412292 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e07cac-f6bc-4cf7-9142-b17f6dea0bae admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:34.529577 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1374/2756] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:24:34 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:24:34.532515 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542938b-3d6b-450d-81fe-5b4f33551428 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:34.542543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542938b-3d6b-450d-81fe-5b4f33551428 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:34.599070 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898331d4-b6d4-4698-9a1d-4471b56e17ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:34.599768 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898331d4-b6d4-4698-9a1d-4471b56e17ef admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:34.600056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898331d4-b6d4-4698-9a1d-4471b56e17ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:34.600331 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898331d4-b6d4-4698-9a1d-4471b56e17ef admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:34.601038 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898331d4-b6d4-4698-9a1d-4471b56e17ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:34.603016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898331d4-b6d4-4698-9a1d-4471b56e17ef admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:34.705132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1383/2757] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:24:34 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:24:34.709109 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e07cac-f6bc-4cf7-9142-b17f6dea0bae admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:34.723485 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e07cac-f6bc-4cf7-9142-b17f6dea0bae admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:34.728332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a897e4e-6878-4b5b-a090-4507c75b8b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:34.729003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a897e4e-6878-4b5b-a090-4507c75b8b1e admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:34.729292 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a897e4e-6878-4b5b-a090-4507c75b8b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:34.729567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a897e4e-6878-4b5b-a090-4507c75b8b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:34.730424 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a897e4e-6878-4b5b-a090-4507c75b8b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:34.732042 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a897e4e-6878-4b5b-a090-4507c75b8b1e admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:34.742754 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1375/2758] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:24:34.746744 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898331d4-b6d4-4698-9a1d-4471b56e17ef admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:34.757183 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898331d4-b6d4-4698-9a1d-4471b56e17ef admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:34.762494 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549071c-aa71-4ede-9838-085b952f3ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:34.763294 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549071c-aa71-4ede-9838-085b952f3ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:34.763517 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549071c-aa71-4ede-9838-085b952f3ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:34.763801 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549071c-aa71-4ede-9838-085b952f3ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:34.764595 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6549071c-aa71-4ede-9838-085b952f3ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:34.766219 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6549071c-aa71-4ede-9838-085b952f3ff8 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:34.780438 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1384/2759] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:24:34.782233 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a897e4e-6878-4b5b-a090-4507c75b8b1e admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:34.793344 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a897e4e-6878-4b5b-a090-4507c75b8b1e admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:34.799000 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ed147-8220-472e-b88c-f81f7eacb541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:34.799977 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ed147-8220-472e-b88c-f81f7eacb541 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:34.800332 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ed147-8220-472e-b88c-f81f7eacb541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:34.800679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ed147-8220-472e-b88c-f81f7eacb541 admin admin] PATH_INFO: `/v3/projects/b5515c98d19e4d53b195f1d190997208/users/e5e4e9e74776483dbb264db2dd26bd17/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:34.816154 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ed147-8220-472e-b88c-f81f7eacb541 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5515c98d19e4d53b195f1d190997208, user_id=e5e4e9e74776483dbb264db2dd26bd17, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:34.819344 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ed147-8220-472e-b88c-f81f7eacb541 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:34.909790 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1376/2760] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:24:34 2022] PUT /identity/v3/projects/b5515c98d19e4d53b195f1d190997208/users/e5e4e9e74776483dbb264db2dd26bd17/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:24:34.920181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf5812-3115-4134-ba32-f51799bd2630 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:34.921300 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf5812-3115-4134-ba32-f51799bd2630 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:34.921749 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf5812-3115-4134-ba32-f51799bd2630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:34.952836 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-03cf5812-3115-4134-ba32-f51799bd2630 None None] MFA Rules not processed for user `e5e4e9e74776483dbb264db2dd26bd17`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:34.954351 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-03cf5812-3115-4134-ba32-f51799bd2630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:35.018665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1385/2761] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:24:35.029416 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ed147-8220-472e-b88c-f81f7eacb541 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:35.031121 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f38ed147-8220-472e-b88c-f81f7eacb541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:35.096028 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ed147-8220-472e-b88c-f81f7eacb541 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:35.101345 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b5f6e-933c-4e90-a7b2-a6f304879f07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:35.102765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b5f6e-933c-4e90-a7b2-a6f304879f07 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:35.103228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b5f6e-933c-4e90-a7b2-a6f304879f07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:35.103509 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b5f6e-933c-4e90-a7b2-a6f304879f07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:35.114585 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b5f6e-933c-4e90-a7b2-a6f304879f07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:35.116075 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b5f6e-933c-4e90-a7b2-a6f304879f07 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:35.126480 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1377/2762] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:24:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:24:36.876284 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7282b-4ed4-4003-9fa1-9e92ad396dfc None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:36.877211 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7282b-4ed4-4003-9fa1-9e92ad396dfc None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:36.877878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7282b-4ed4-4003-9fa1-9e92ad396dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:36.900373 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-4dc7282b-4ed4-4003-9fa1-9e92ad396dfc None None] MFA Rules not processed for user `e5e4e9e74776483dbb264db2dd26bd17`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:36.901902 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4dc7282b-4ed4-4003-9fa1-9e92ad396dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:36.912598 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1386/2763] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:24:40.497274 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f8aa1-4ce1-4fcf-b3b7-13c114df400f None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:40.497803 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f8aa1-4ce1-4fcf-b3b7-13c114df400f None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:40.498125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f8aa1-4ce1-4fcf-b3b7-13c114df400f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:40.534149 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-db8f8aa1-4ce1-4fcf-b3b7-13c114df400f None None] MFA Rules not processed for user `04ce1bdd6a234177ba7745d19d6fd950`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:40.535464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-db8f8aa1-4ce1-4fcf-b3b7-13c114df400f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:40.669958 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1378/2764] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:24:40.687680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7282b-4ed4-4003-9fa1-9e92ad396dfc None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:40.689306 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4dc7282b-4ed4-4003-9fa1-9e92ad396dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:40.792898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7282b-4ed4-4003-9fa1-9e92ad396dfc None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:40.799975 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c23bc9-6606-48f6-831d-9d31bb2756f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:40.801342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c23bc9-6606-48f6-831d-9d31bb2756f0 service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:40.802008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c23bc9-6606-48f6-831d-9d31bb2756f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:40.802571 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c23bc9-6606-48f6-831d-9d31bb2756f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:40.812230 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c23bc9-6606-48f6-831d-9d31bb2756f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:40.818251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c23bc9-6606-48f6-831d-9d31bb2756f0 service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:40.826528 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1387/2765] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:24:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:24:44.161411 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150ac5f-df9b-44f0-91a3-cc79b7b83697 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:44.162300 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150ac5f-df9b-44f0-91a3-cc79b7b83697 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:44.162921 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150ac5f-df9b-44f0-91a3-cc79b7b83697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:44.184615 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-a150ac5f-df9b-44f0-91a3-cc79b7b83697 None None] MFA Rules not processed for user `04ce1bdd6a234177ba7745d19d6fd950`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:24:44.186572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a150ac5f-df9b-44f0-91a3-cc79b7b83697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:44.210275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1379/2766] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:24:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:24:46.312751 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c23bc9-6606-48f6-831d-9d31bb2756f0 service nova] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:46.314198 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-47c23bc9-6606-48f6-831d-9d31bb2756f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:46.381039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c23bc9-6606-48f6-831d-9d31bb2756f0 service nova] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:46.386747 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ceed41-a1ca-4491-bd5a-d22f6b3487bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022093dea10d46478b3016395c712d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:46.387383 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ceed41-a1ca-4491-bd5a-d22f6b3487bf service placement] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:46.387668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ceed41-a1ca-4491-bd5a-d22f6b3487bf service placement] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:46.387948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ceed41-a1ca-4491-bd5a-d22f6b3487bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:46.389774 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-47ceed41-a1ca-4491-bd5a-d22f6b3487bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:46.401326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ceed41-a1ca-4491-bd5a-d22f6b3487bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:46.402964 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ceed41-a1ca-4491-bd5a-d22f6b3487bf service placement] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:46.413317 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1388/2767] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:24:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:24:54.210643 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150ac5f-df9b-44f0-91a3-cc79b7b83697 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:24:54.212691 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a150ac5f-df9b-44f0-91a3-cc79b7b83697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:54.268096 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150ac5f-df9b-44f0-91a3-cc79b7b83697 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:24:54.271799 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba7db1-b898-47cf-9242-88e4ef7510fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:24:54.272475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba7db1-b898-47cf-9242-88e4ef7510fe service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:24:54.272758 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba7db1-b898-47cf-9242-88e4ef7510fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:24:54.273037 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba7db1-b898-47cf-9242-88e4ef7510fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:24:54.275532 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-3cba7db1-b898-47cf-9242-88e4ef7510fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:24:54.330761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba7db1-b898-47cf-9242-88e4ef7510fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:24:54.332183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba7db1-b898-47cf-9242-88e4ef7510fe service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:24:54.341839 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1380/2768] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:24:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:25:02.184337 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b621829-37b8-4074-b6a0-430bb9e25ef9 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:25:02.188285 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b621829-37b8-4074-b6a0-430bb9e25ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:25:02.188285 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b621829-37b8-4074-b6a0-430bb9e25ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:25:02.203079 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-8b621829-37b8-4074-b6a0-430bb9e25ef9 None None] MFA Rules not processed for user `7ec2605fd1b84e0ea134b943ed33a605`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:25:02.204328 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-8b621829-37b8-4074-b6a0-430bb9e25ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:25:02.259472 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1389/2769] 10.222.0.65 () {58 vars in 1009 bytes} [Tue May 10 20:25:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:25:02.407014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba7db1-b898-47cf-9242-88e4ef7510fe service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:25:02.408479 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-3cba7db1-b898-47cf-9242-88e4ef7510fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:25:02.477376 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba7db1-b898-47cf-9242-88e4ef7510fe service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:25:02.482420 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe3351-d24a-465d-9e97-2d589a63f0d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d10b258a53484da22950e2b1db2e41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:25:02.483067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe3351-d24a-465d-9e97-2d589a63f0d5 service swift] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:25:02.483379 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe3351-d24a-465d-9e97-2d589a63f0d5 service swift] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:25:02.483655 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe3351-d24a-465d-9e97-2d589a63f0d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:25:02.491752 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe3351-d24a-465d-9e97-2d589a63f0d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:25:02.493380 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe3351-d24a-465d-9e97-2d589a63f0d5 service swift] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:25:02.501277 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1381/2770] 10.222.0.65 () {60 vars in 1384 bytes} [Tue May 10 20:25:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 20:25:03.277192 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3f6b8-f574-4782-80d8-e84eb7f2d959 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:25:03.277714 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3f6b8-f574-4782-80d8-e84eb7f2d959 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:25:03.277849 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3f6b8-f574-4782-80d8-e84eb7f2d959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:25:03.297213 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-7ef3f6b8-f574-4782-80d8-e84eb7f2d959 None None] MFA Rules not processed for user `7ec2605fd1b84e0ea134b943ed33a605`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:25:03.298495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7ef3f6b8-f574-4782-80d8-e84eb7f2d959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:25:03.305858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1390/2771] 10.222.0.65 () {58 vars in 967 bytes} [Tue May 10 20:25:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:25:03.318505 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe3351-d24a-465d-9e97-2d589a63f0d5 service swift] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:25:03.329836 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe3351-d24a-465d-9e97-2d589a63f0d5 service swift] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:25:03.335074 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45436d47-648e-40ff-a3cd-3db2d78a1ec0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d10b258a53484da22950e2b1db2e41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:25:03.335724 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45436d47-648e-40ff-a3cd-3db2d78a1ec0 service swift] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:25:03.336007 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45436d47-648e-40ff-a3cd-3db2d78a1ec0 service swift] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:25:03.337190 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45436d47-648e-40ff-a3cd-3db2d78a1ec0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:25:03.345103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45436d47-648e-40ff-a3cd-3db2d78a1ec0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:25:03.346697 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45436d47-648e-40ff-a3cd-3db2d78a1ec0 service swift] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:25:03.354454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1382/2772] 10.222.0.65 () {60 vars in 1384 bytes} [Tue May 10 20:25:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 20:25:03.969581 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be5bec-50d0-42c9-bc91-792e6e286e1a None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:25:03.969581 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be5bec-50d0-42c9-bc91-792e6e286e1a None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:25:03.969581 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be5bec-50d0-42c9-bc91-792e6e286e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:25:03.987247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-25be5bec-50d0-42c9-bc91-792e6e286e1a None None] MFA Rules not processed for user `7ec2605fd1b84e0ea134b943ed33a605`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:25:03.988469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-25be5bec-50d0-42c9-bc91-792e6e286e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:25:03.996601 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1391/2773] 10.222.0.65 () {58 vars in 1009 bytes} [Tue May 10 20:25:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:25:04.008315 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45436d47-648e-40ff-a3cd-3db2d78a1ec0 service swift] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:25:04.018735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45436d47-648e-40ff-a3cd-3db2d78a1ec0 service swift] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:25:04.023788 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd191752-4fda-4923-aa70-ba30b281b975 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d10b258a53484da22950e2b1db2e41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:25:04.024416 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd191752-4fda-4923-aa70-ba30b281b975 service swift] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:25:04.024703 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd191752-4fda-4923-aa70-ba30b281b975 service swift] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:25:04.024982 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd191752-4fda-4923-aa70-ba30b281b975 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:25:04.033184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd191752-4fda-4923-aa70-ba30b281b975 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:25:04.034806 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd191752-4fda-4923-aa70-ba30b281b975 service swift] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:25:04.043494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1383/2774] 10.222.0.65 () {60 vars in 1384 bytes} [Tue May 10 20:25:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 20:25:04.656329 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe88101-1521-4d6f-a065-4191f90c4ac6 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:25:04.656780 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe88101-1521-4d6f-a065-4191f90c4ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:25:04.657371 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe88101-1521-4d6f-a065-4191f90c4ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:25:04.674405 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-cfe88101-1521-4d6f-a065-4191f90c4ac6 None None] MFA Rules not processed for user `7ec2605fd1b84e0ea134b943ed33a605`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:25:04.675577 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-cfe88101-1521-4d6f-a065-4191f90c4ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:25:04.682704 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1392/2775] 10.222.0.65 () {58 vars in 967 bytes} [Tue May 10 20:25:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:25:04.694625 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd191752-4fda-4923-aa70-ba30b281b975 service swift] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:25:04.705755 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd191752-4fda-4923-aa70-ba30b281b975 service swift] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:25:04.710777 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b183ca-8c54-4195-ad89-388ec3bb9067 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d10b258a53484da22950e2b1db2e41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:25:04.712003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b183ca-8c54-4195-ad89-388ec3bb9067 service swift] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:25:04.712286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b183ca-8c54-4195-ad89-388ec3bb9067 service swift] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:25:04.712564 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b183ca-8c54-4195-ad89-388ec3bb9067 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:25:04.720562 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b183ca-8c54-4195-ad89-388ec3bb9067 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:25:04.722150 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b183ca-8c54-4195-ad89-388ec3bb9067 service swift] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:25:04.730049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1384/2776] 10.222.0.65 () {60 vars in 1384 bytes} [Tue May 10 20:25:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 20:25:23.418032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe88101-1521-4d6f-a065-4191f90c4ac6 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:25:23.427338 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe88101-1521-4d6f-a065-4191f90c4ac6 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:25:23.431616 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db9680-a2ca-4ca9-8866-f34372595c63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:25:23.432513 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db9680-a2ca-4ca9-8866-f34372595c63 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:25:23.432513 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db9680-a2ca-4ca9-8866-f34372595c63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:25:23.432794 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db9680-a2ca-4ca9-8866-f34372595c63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:25:23.434604 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-87db9680-a2ca-4ca9-8866-f34372595c63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:25:23.491577 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db9680-a2ca-4ca9-8866-f34372595c63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:25:23.493670 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db9680-a2ca-4ca9-8866-f34372595c63 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:25:23.502417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1393/2777] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:25:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:26:38.192531 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b183ca-8c54-4195-ad89-388ec3bb9067 service swift] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:26:38.202171 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b183ca-8c54-4195-ad89-388ec3bb9067 service swift] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:26:38.206006 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833c88d-5433-407c-8d09-97a0b9ca992e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:26:38.206680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833c88d-5433-407c-8d09-97a0b9ca992e service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:26:38.206961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833c88d-5433-407c-8d09-97a0b9ca992e service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:26:38.207238 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833c88d-5433-407c-8d09-97a0b9ca992e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:26:38.208987 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-e833c88d-5433-407c-8d09-97a0b9ca992e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:26:38.222693 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833c88d-5433-407c-8d09-97a0b9ca992e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:26:38.224105 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833c88d-5433-407c-8d09-97a0b9ca992e service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:26:38.234798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1385/2778] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:26:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:26:41.541074 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db9680-a2ca-4ca9-8866-f34372595c63 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:26:41.546291 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-87db9680-a2ca-4ca9-8866-f34372595c63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:26:41.554797 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db9680-a2ca-4ca9-8866-f34372595c63 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:26:41.558990 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc655b0-273a-4776-bc43-92f66f996405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:26:41.559669 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc655b0-273a-4776-bc43-92f66f996405 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:26:41.559953 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc655b0-273a-4776-bc43-92f66f996405 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:26:41.560231 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc655b0-273a-4776-bc43-92f66f996405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:26:41.562103 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3fc655b0-273a-4776-bc43-92f66f996405 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:26:41.575723 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc655b0-273a-4776-bc43-92f66f996405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:26:41.577150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc655b0-273a-4776-bc43-92f66f996405 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:26:41.589123 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1394/2779] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:26:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:27:43.171495 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833c88d-5433-407c-8d09-97a0b9ca992e service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:27:43.188122 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833c88d-5433-407c-8d09-97a0b9ca992e service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:27:43.192270 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67306e05-5437-418f-bd49-61b94725c62c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:27:43.192915 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67306e05-5437-418f-bd49-61b94725c62c service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:27:43.193198 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67306e05-5437-418f-bd49-61b94725c62c service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:27:43.193476 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67306e05-5437-418f-bd49-61b94725c62c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:27:43.195444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-67306e05-5437-418f-bd49-61b94725c62c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:27:43.205367 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67306e05-5437-418f-bd49-61b94725c62c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:27:43.206826 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67306e05-5437-418f-bd49-61b94725c62c service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:27:43.214992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1386/2780] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:27:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:28:05.472448 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc655b0-273a-4776-bc43-92f66f996405 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:28:05.473829 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3fc655b0-273a-4776-bc43-92f66f996405 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:28:05.485536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc655b0-273a-4776-bc43-92f66f996405 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:28:05.488998 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab2eb0-8d86-431a-aac9-852dc23e6d40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:28:05.489617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab2eb0-8d86-431a-aac9-852dc23e6d40 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:28:05.489897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab2eb0-8d86-431a-aac9-852dc23e6d40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:28:05.490195 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab2eb0-8d86-431a-aac9-852dc23e6d40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:28:05.491864 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-8aab2eb0-8d86-431a-aac9-852dc23e6d40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:28:05.547540 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab2eb0-8d86-431a-aac9-852dc23e6d40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:28:05.548956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab2eb0-8d86-431a-aac9-852dc23e6d40 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:28:05.557594 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1395/2781] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:28:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:29:35.481947 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67306e05-5437-418f-bd49-61b94725c62c service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:29:35.528830 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67306e05-5437-418f-bd49-61b94725c62c service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:29:35.532399 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccf6c0-d1c7-4290-8b69-7b63da678f76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:29:35.533030 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccf6c0-d1c7-4290-8b69-7b63da678f76 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:29:35.533306 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccf6c0-d1c7-4290-8b69-7b63da678f76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:29:35.533574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccf6c0-d1c7-4290-8b69-7b63da678f76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:29:35.554487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ccf6c0-d1c7-4290-8b69-7b63da678f76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:29:35.555897 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ccf6c0-d1c7-4290-8b69-7b63da678f76 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:29:35.565054 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1387/2782] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:29:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:29:40.664051 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab2eb0-8d86-431a-aac9-852dc23e6d40 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:29:40.692321 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab2eb0-8d86-431a-aac9-852dc23e6d40 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:29:40.695930 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e397a4-1f22-44a1-b840-60f16255434b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:29:40.696566 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e397a4-1f22-44a1-b840-60f16255434b service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:29:40.696875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e397a4-1f22-44a1-b840-60f16255434b service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:29:40.697150 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e397a4-1f22-44a1-b840-60f16255434b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:29:40.705603 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e397a4-1f22-44a1-b840-60f16255434b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:29:40.707050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e397a4-1f22-44a1-b840-60f16255434b service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:29:40.715595 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1396/2783] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:29:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:29:54.382881 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccf6c0-d1c7-4290-8b69-7b63da678f76 service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:29:54.384312 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-14ccf6c0-d1c7-4290-8b69-7b63da678f76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:29:54.447704 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccf6c0-d1c7-4290-8b69-7b63da678f76 service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:29:54.454871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96d83e-d42e-4e29-a7a8-d41e34e73ab5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:29:54.455522 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96d83e-d42e-4e29-a7a8-d41e34e73ab5 service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:29:54.455803 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96d83e-d42e-4e29-a7a8-d41e34e73ab5 service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:29:54.456074 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96d83e-d42e-4e29-a7a8-d41e34e73ab5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:29:54.464753 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96d83e-d42e-4e29-a7a8-d41e34e73ab5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:29:54.466370 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96d83e-d42e-4e29-a7a8-d41e34e73ab5 service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:29:54.493743 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1388/2784] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:29:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:30:22.713306 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e397a4-1f22-44a1-b840-60f16255434b service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:30:22.722350 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e397a4-1f22-44a1-b840-60f16255434b service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:30:22.725879 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff930c-8230-404b-95e6-89d593bdb40f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:30:22.726540 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff930c-8230-404b-95e6-89d593bdb40f service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:30:22.726824 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff930c-8230-404b-95e6-89d593bdb40f service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:30:22.727098 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff930c-8230-404b-95e6-89d593bdb40f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:30:22.735245 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff930c-8230-404b-95e6-89d593bdb40f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:30:22.736653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff930c-8230-404b-95e6-89d593bdb40f service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:30:22.763418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1397/2785] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:30:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:30:48.002739 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96d83e-d42e-4e29-a7a8-d41e34e73ab5 service nova] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:30:48.004103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-6e96d83e-d42e-4e29-a7a8-d41e34e73ab5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:30:48.039484 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96d83e-d42e-4e29-a7a8-d41e34e73ab5 service nova] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:30:48.044436 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83560e61-219d-40b2-a68d-484f442ebf0e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022093dea10d46478b3016395c712d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:30:48.045071 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83560e61-219d-40b2-a68d-484f442ebf0e service placement] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:30:48.045351 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83560e61-219d-40b2-a68d-484f442ebf0e service placement] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:30:48.045627 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83560e61-219d-40b2-a68d-484f442ebf0e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:30:48.053871 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83560e61-219d-40b2-a68d-484f442ebf0e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:30:48.055472 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83560e61-219d-40b2-a68d-484f442ebf0e service placement] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:30:48.065565 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1389/2786] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:30:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:31:01.598665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff930c-8230-404b-95e6-89d593bdb40f service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:31:01.618658 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff930c-8230-404b-95e6-89d593bdb40f service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:31:01.623092 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85359ddf-55aa-4b45-9055-719520125b6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:31:01.623766 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85359ddf-55aa-4b45-9055-719520125b6d service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:31:01.624054 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85359ddf-55aa-4b45-9055-719520125b6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:31:01.624334 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85359ddf-55aa-4b45-9055-719520125b6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:31:01.626141 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-85359ddf-55aa-4b45-9055-719520125b6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:31:01.708534 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85359ddf-55aa-4b45-9055-719520125b6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:31:01.711323 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85359ddf-55aa-4b45-9055-719520125b6d service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:31:01.728319 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1398/2787] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:31:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:31:09.075712 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83560e61-219d-40b2-a68d-484f442ebf0e service placement] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:31:09.077114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-83560e61-219d-40b2-a68d-484f442ebf0e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:31:09.091245 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83560e61-219d-40b2-a68d-484f442ebf0e service placement] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:31:09.096160 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ac98e-ae23-45a1-a99f-85c2155b4624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:31:09.097038 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ac98e-ae23-45a1-a99f-85c2155b4624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:31:09.097327 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ac98e-ae23-45a1-a99f-85c2155b4624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:31:09.097604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ac98e-ae23-45a1-a99f-85c2155b4624 admin admin] PATH_INFO: `/v3/users/f213f79341ab4462b3d51352ccb9de2c` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:31:09.111940 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ac98e-ae23-45a1-a99f-85c2155b4624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f213f79341ab4462b3d51352ccb9de2c)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:31:09.113861 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ac98e-ae23-45a1-a99f-85c2155b4624 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:31:09.305103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1f3ac98e-ae23-45a1-a99f-85c2155b4624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f213f79341ab4462b3d51352ccb9de2c'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:09.305814 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1f3ac98e-ae23-45a1-a99f-85c2155b4624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f213f79341ab4462b3d51352ccb9de2c'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:09.319974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1f3ac98e-ae23-45a1-a99f-85c2155b4624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f213f79341ab4462b3d51352ccb9de2c'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:09.320635 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1f3ac98e-ae23-45a1-a99f-85c2155b4624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f213f79341ab4462b3d51352ccb9de2c'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:09.331526 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-1f3ac98e-ae23-45a1-a99f-85c2155b4624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f213f79341ab4462b3d51352ccb9de2c'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:09.428930 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1390/2788] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:31:09 2022] DELETE /identity/v3/users/f213f79341ab4462b3d51352ccb9de2c => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:31:09.682251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85359ddf-55aa-4b45-9055-719520125b6d service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:31:09.692703 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85359ddf-55aa-4b45-9055-719520125b6d service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:31:09.750199 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14e34e-a425-495d-953c-f03444536c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:31:09.751124 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14e34e-a425-495d-953c-f03444536c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:31:09.751409 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14e34e-a425-495d-953c-f03444536c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:31:09.751687 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14e34e-a425-495d-953c-f03444536c7e admin admin] PATH_INFO: `/v3/projects/b9724c9a575141d48bbfbde685774622` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:31:09.752870 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14e34e-a425-495d-953c-f03444536c7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9724c9a575141d48bbfbde685774622)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:31:09.754826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14e34e-a425-495d-953c-f03444536c7e admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:31:09.885921 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6e14e34e-a425-495d-953c-f03444536c7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9724c9a575141d48bbfbde685774622', 'request_id': 'req-6e14e34e-a425-495d-953c-f03444536c7e'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:09.971596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6e14e34e-a425-495d-953c-f03444536c7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9724c9a575141d48bbfbde685774622', 'request_id': 'req-6e14e34e-a425-495d-953c-f03444536c7e'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:09.977563 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6e14e34e-a425-495d-953c-f03444536c7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9724c9a575141d48bbfbde685774622', 'request_id': 'req-6e14e34e-a425-495d-953c-f03444536c7e'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:09.988718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6e14e34e-a425-495d-953c-f03444536c7e admin admin] The token cache is being invalidate because project b9724c9a575141d48bbfbde685774622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:31:09.988999 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6e14e34e-a425-495d-953c-f03444536c7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:09.989621 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-6e14e34e-a425-495d-953c-f03444536c7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:09.991187 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1399/2789] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:31:09 2022] DELETE /identity/v3/projects/b9724c9a575141d48bbfbde685774622 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:31:09.995311 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ac98e-ae23-45a1-a99f-85c2155b4624 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:31:09.996921 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-1f3ac98e-ae23-45a1-a99f-85c2155b4624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:31:10.010042 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ac98e-ae23-45a1-a99f-85c2155b4624 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:31:10.066637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692af5bd-48ff-4142-a5af-67b13c37cb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:31:10.067513 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692af5bd-48ff-4142-a5af-67b13c37cb15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:31:10.067794 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692af5bd-48ff-4142-a5af-67b13c37cb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:31:10.068066 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692af5bd-48ff-4142-a5af-67b13c37cb15 admin admin] PATH_INFO: `/v3/users/c91839388e054c0cbe7c8e7bbb35c240` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:31:10.083135 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692af5bd-48ff-4142-a5af-67b13c37cb15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c91839388e054c0cbe7c8e7bbb35c240)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:31:10.085049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692af5bd-48ff-4142-a5af-67b13c37cb15 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:31:10.299173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-692af5bd-48ff-4142-a5af-67b13c37cb15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c91839388e054c0cbe7c8e7bbb35c240'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:10.299725 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-692af5bd-48ff-4142-a5af-67b13c37cb15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c91839388e054c0cbe7c8e7bbb35c240'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:10.311425 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-692af5bd-48ff-4142-a5af-67b13c37cb15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c91839388e054c0cbe7c8e7bbb35c240'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:10.312041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-692af5bd-48ff-4142-a5af-67b13c37cb15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c91839388e054c0cbe7c8e7bbb35c240'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:10.318671 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-692af5bd-48ff-4142-a5af-67b13c37cb15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c91839388e054c0cbe7c8e7bbb35c240'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:10.388711 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1391/2790] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:31:09 2022] DELETE /identity/v3/users/c91839388e054c0cbe7c8e7bbb35c240 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:31:10.596306 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14e34e-a425-495d-953c-f03444536c7e admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:31:10.597887 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-6e14e34e-a425-495d-953c-f03444536c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:31:10.611455 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14e34e-a425-495d-953c-f03444536c7e admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:31:10.666464 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1eecd8-850b-4dc1-ae5f-6df7a143cebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:31:10.667812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1eecd8-850b-4dc1-ae5f-6df7a143cebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:31:10.668088 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1eecd8-850b-4dc1-ae5f-6df7a143cebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:31:10.668358 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1eecd8-850b-4dc1-ae5f-6df7a143cebf admin admin] PATH_INFO: `/v3/projects/ef25570884a043a9811f92c3391ec916` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:31:10.678383 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1eecd8-850b-4dc1-ae5f-6df7a143cebf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef25570884a043a9811f92c3391ec916)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:31:10.681344 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1eecd8-850b-4dc1-ae5f-6df7a143cebf admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:31:10.764676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-db1eecd8-850b-4dc1-ae5f-6df7a143cebf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef25570884a043a9811f92c3391ec916', 'request_id': 'req-db1eecd8-850b-4dc1-ae5f-6df7a143cebf'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:10.822863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-db1eecd8-850b-4dc1-ae5f-6df7a143cebf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef25570884a043a9811f92c3391ec916', 'request_id': 'req-db1eecd8-850b-4dc1-ae5f-6df7a143cebf'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:10.828743 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-db1eecd8-850b-4dc1-ae5f-6df7a143cebf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef25570884a043a9811f92c3391ec916', 'request_id': 'req-db1eecd8-850b-4dc1-ae5f-6df7a143cebf'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:10.840122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-db1eecd8-850b-4dc1-ae5f-6df7a143cebf admin admin] The token cache is being invalidate because project ef25570884a043a9811f92c3391ec916 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:31:10.840346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-db1eecd8-850b-4dc1-ae5f-6df7a143cebf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:10.840968 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-db1eecd8-850b-4dc1-ae5f-6df7a143cebf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:31:10.842538 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1400/2791] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:31:10 2022] DELETE /identity/v3/projects/ef25570884a043a9811f92c3391ec916 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:32:48.163789 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692af5bd-48ff-4142-a5af-67b13c37cb15 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:32:48.165265 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-692af5bd-48ff-4142-a5af-67b13c37cb15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:32:48.233567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692af5bd-48ff-4142-a5af-67b13c37cb15 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:32:48.279867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42a37f-688a-425d-82fe-431df2dda71b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:32:48.280517 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42a37f-688a-425d-82fe-431df2dda71b service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:32:48.280833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42a37f-688a-425d-82fe-431df2dda71b service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:32:48.281111 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42a37f-688a-425d-82fe-431df2dda71b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:32:48.283003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0c42a37f-688a-425d-82fe-431df2dda71b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:32:48.292154 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42a37f-688a-425d-82fe-431df2dda71b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:32:48.293574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42a37f-688a-425d-82fe-431df2dda71b service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:32:48.301961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1392/2792] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:32:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:32:59.563256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1eecd8-850b-4dc1-ae5f-6df7a143cebf admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:32:59.565307 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-db1eecd8-850b-4dc1-ae5f-6df7a143cebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:32:59.642757 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1eecd8-850b-4dc1-ae5f-6df7a143cebf admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:32:59.647934 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a975cc-d073-440e-80dc-814d1211acec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:32:59.648847 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a975cc-d073-440e-80dc-814d1211acec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:32:59.649132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a975cc-d073-440e-80dc-814d1211acec admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:32:59.649409 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a975cc-d073-440e-80dc-814d1211acec admin admin] PATH_INFO: `/v3/users/e5e4e9e74776483dbb264db2dd26bd17` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:32:59.650653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a975cc-d073-440e-80dc-814d1211acec admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5e4e9e74776483dbb264db2dd26bd17)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:32:59.652611 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a975cc-d073-440e-80dc-814d1211acec admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:32:59.851641 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-f7a975cc-d073-440e-80dc-814d1211acec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5e4e9e74776483dbb264db2dd26bd17'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:32:59.852212 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-f7a975cc-d073-440e-80dc-814d1211acec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5e4e9e74776483dbb264db2dd26bd17'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:32:59.852807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-f7a975cc-d073-440e-80dc-814d1211acec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5e4e9e74776483dbb264db2dd26bd17'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:32:59.864591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-f7a975cc-d073-440e-80dc-814d1211acec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5e4e9e74776483dbb264db2dd26bd17'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:32:59.871332 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-f7a975cc-d073-440e-80dc-814d1211acec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5e4e9e74776483dbb264db2dd26bd17'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:32:59.947347 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42a37f-688a-425d-82fe-431df2dda71b service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:32:59.948850 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0c42a37f-688a-425d-82fe-431df2dda71b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:32:59.960216 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42a37f-688a-425d-82fe-431df2dda71b service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:32:59.964574 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e8a18-60dc-4e34-9818-d7328f80a075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:32:59.965226 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e8a18-60dc-4e34-9818-d7328f80a075 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:32:59.966074 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e8a18-60dc-4e34-9818-d7328f80a075 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:32:59.966455 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e8a18-60dc-4e34-9818-d7328f80a075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:32:59.968710 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f23e8a18-60dc-4e34-9818-d7328f80a075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:32:59.970922 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1401/2793] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:32:59 2022] DELETE /identity/v3/users/e5e4e9e74776483dbb264db2dd26bd17 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:32:59.975328 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b23b57-0faf-409e-890d-5b3a15a07a80 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:32:59.975737 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b23b57-0faf-409e-890d-5b3a15a07a80 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:32:59.976023 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b23b57-0faf-409e-890d-5b3a15a07a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:33:00.002678 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-10b23b57-0faf-409e-890d-5b3a15a07a80 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:33:00.003880 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-10b23b57-0faf-409e-890d-5b3a15a07a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:33:00.042179 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e8a18-60dc-4e34-9818-d7328f80a075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:33:00.043650 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e8a18-60dc-4e34-9818-d7328f80a075 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:33:00.054493 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1393/2794] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:32:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:33:00.060748 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1402/2795] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:32:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:33:00.071213 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e8a18-60dc-4e34-9818-d7328f80a075 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:33:00.072893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f23e8a18-60dc-4e34-9818-d7328f80a075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:33:00.084046 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e8a18-60dc-4e34-9818-d7328f80a075 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:33:00.131365 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fe221-cd29-48a1-9ddb-f8ca7d8d78b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:33:00.132014 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fe221-cd29-48a1-9ddb-f8ca7d8d78b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:33:00.132295 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fe221-cd29-48a1-9ddb-f8ca7d8d78b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:33:00.132567 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fe221-cd29-48a1-9ddb-f8ca7d8d78b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:33:00.143078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fe221-cd29-48a1-9ddb-f8ca7d8d78b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:33:00.144553 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fe221-cd29-48a1-9ddb-f8ca7d8d78b0 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:33:00.154234 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1394/2796] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:33:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:33:00.399470 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b23b57-0faf-409e-890d-5b3a15a07a80 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:33:00.402341 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-10b23b57-0faf-409e-890d-5b3a15a07a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:33:00.412532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b23b57-0faf-409e-890d-5b3a15a07a80 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:33:00.421754 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e702bdc-3117-4949-bcc6-85cdaa3f2425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:33:00.422682 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e702bdc-3117-4949-bcc6-85cdaa3f2425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:33:00.422971 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e702bdc-3117-4949-bcc6-85cdaa3f2425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:33:00.423887 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e702bdc-3117-4949-bcc6-85cdaa3f2425 admin admin] PATH_INFO: `/v3/projects/b5515c98d19e4d53b195f1d190997208` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:33:00.424525 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e702bdc-3117-4949-bcc6-85cdaa3f2425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5515c98d19e4d53b195f1d190997208)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:33:00.427491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e702bdc-3117-4949-bcc6-85cdaa3f2425 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:33:00.540921 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1e702bdc-3117-4949-bcc6-85cdaa3f2425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5515c98d19e4d53b195f1d190997208', 'request_id': 'req-1e702bdc-3117-4949-bcc6-85cdaa3f2425'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:33:00.592480 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1e702bdc-3117-4949-bcc6-85cdaa3f2425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5515c98d19e4d53b195f1d190997208', 'request_id': 'req-1e702bdc-3117-4949-bcc6-85cdaa3f2425'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:33:00.598534 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1e702bdc-3117-4949-bcc6-85cdaa3f2425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5515c98d19e4d53b195f1d190997208', 'request_id': 'req-1e702bdc-3117-4949-bcc6-85cdaa3f2425'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:33:00.610235 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1e702bdc-3117-4949-bcc6-85cdaa3f2425 admin admin] The token cache is being invalidate because project b5515c98d19e4d53b195f1d190997208 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:33:00.610525 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1e702bdc-3117-4949-bcc6-85cdaa3f2425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:33:00.611160 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1e702bdc-3117-4949-bcc6-85cdaa3f2425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:33:00.613025 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1403/2797] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:33:00 2022] DELETE /identity/v3/projects/b5515c98d19e4d53b195f1d190997208 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:33:01.332962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fe221-cd29-48a1-9ddb-f8ca7d8d78b0 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:33:01.334547 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-5d6fe221-cd29-48a1-9ddb-f8ca7d8d78b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:33:01.419485 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fe221-cd29-48a1-9ddb-f8ca7d8d78b0 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:33:01.498386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8ee94-a533-426c-8f81-e6e80f827157 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:33:01.500618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8ee94-a533-426c-8f81-e6e80f827157 service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:33:01.500618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8ee94-a533-426c-8f81-e6e80f827157 service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:33:01.500618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8ee94-a533-426c-8f81-e6e80f827157 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:33:01.502231 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-deb8ee94-a533-426c-8f81-e6e80f827157 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:33:01.519442 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8ee94-a533-426c-8f81-e6e80f827157 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:33:01.521045 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8ee94-a533-426c-8f81-e6e80f827157 service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:33:01.534115 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1395/2798] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:33:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:33:05.325502 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2b2f7-82dd-4971-bf3d-1a24bb0733d3 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:33:05.327997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2b2f7-82dd-4971-bf3d-1a24bb0733d3 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:33:05.327997 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2b2f7-82dd-4971-bf3d-1a24bb0733d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:33:05.345244 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-30e2b2f7-82dd-4971-bf3d-1a24bb0733d3 None None] MFA Rules not processed for user `04ce1bdd6a234177ba7745d19d6fd950`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:33:05.346333 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-30e2b2f7-82dd-4971-bf3d-1a24bb0733d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:33:05.397761 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1404/2799] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:33:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:33:05.691844 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8ee94-a533-426c-8f81-e6e80f827157 service nova] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:33:05.693268 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-deb8ee94-a533-426c-8f81-e6e80f827157 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:33:05.749209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8ee94-a533-426c-8f81-e6e80f827157 service nova] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:33:05.752822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a083e2b-9da4-4378-bd70-7f3052fe1b64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:33:05.753464 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a083e2b-9da4-4378-bd70-7f3052fe1b64 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:33:05.753746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a083e2b-9da4-4378-bd70-7f3052fe1b64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:33:05.754021 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a083e2b-9da4-4378-bd70-7f3052fe1b64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:33:05.755801 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8a083e2b-9da4-4378-bd70-7f3052fe1b64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:33:05.765556 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a083e2b-9da4-4378-bd70-7f3052fe1b64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:33:05.767000 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a083e2b-9da4-4378-bd70-7f3052fe1b64 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:33:05.774864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1396/2800] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:33:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:33:06.167336 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2b2f7-82dd-4971-bf3d-1a24bb0733d3 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:33:06.168796 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-30e2b2f7-82dd-4971-bf3d-1a24bb0733d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:33:06.237633 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2b2f7-82dd-4971-bf3d-1a24bb0733d3 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:33:06.242617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e979b7b-7cd6-4094-b01d-3093a442b931 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022093dea10d46478b3016395c712d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:33:06.243260 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e979b7b-7cd6-4094-b01d-3093a442b931 service placement] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:33:06.243540 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e979b7b-7cd6-4094-b01d-3093a442b931 service placement] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:33:06.243814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e979b7b-7cd6-4094-b01d-3093a442b931 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:33:06.245564 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-8e979b7b-7cd6-4094-b01d-3093a442b931 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:33:06.257217 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e979b7b-7cd6-4094-b01d-3093a442b931 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:33:06.258813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e979b7b-7cd6-4094-b01d-3093a442b931 service placement] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:33:06.269078 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1405/2801] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:33:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:33:16.034864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a083e2b-9da4-4378-bd70-7f3052fe1b64 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:33:16.036315 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-8a083e2b-9da4-4378-bd70-7f3052fe1b64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:33:16.053600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a083e2b-9da4-4378-bd70-7f3052fe1b64 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:33:16.057461 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0c284-e6bf-4959-9d12-982962ff55f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:33:16.058114 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0c284-e6bf-4959-9d12-982962ff55f8 service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:33:16.058402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0c284-e6bf-4959-9d12-982962ff55f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:33:16.058674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0c284-e6bf-4959-9d12-982962ff55f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:33:16.060917 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c9b0c284-e6bf-4959-9d12-982962ff55f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:33:16.073020 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0c284-e6bf-4959-9d12-982962ff55f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:33:16.074607 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0c284-e6bf-4959-9d12-982962ff55f8 service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:33:16.085603 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1397/2802] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:33:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:34:54.203021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e979b7b-7cd6-4094-b01d-3093a442b931 service placement] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:34:54.215743 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e979b7b-7cd6-4094-b01d-3093a442b931 service placement] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:34:54.221267 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518d0ff-02ca-44c4-b972-668832368fe5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:34:54.222490 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518d0ff-02ca-44c4-b972-668832368fe5 service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:34:54.222779 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518d0ff-02ca-44c4-b972-668832368fe5 service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:34:54.223059 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518d0ff-02ca-44c4-b972-668832368fe5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:34:54.224814 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-c518d0ff-02ca-44c4-b972-668832368fe5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:34:54.256844 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518d0ff-02ca-44c4-b972-668832368fe5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:34:54.258442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518d0ff-02ca-44c4-b972-668832368fe5 service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:34:54.276589 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1406/2803] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:34:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:34:59.672516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0c284-e6bf-4959-9d12-982962ff55f8 service nova] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:34:59.673987 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c9b0c284-e6bf-4959-9d12-982962ff55f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:34:59.741931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0c284-e6bf-4959-9d12-982962ff55f8 service nova] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:34:59.745576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b14324-c596-42d9-b88b-7420642a5484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:34:59.746262 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b14324-c596-42d9-b88b-7420642a5484 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:34:59.746555 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b14324-c596-42d9-b88b-7420642a5484 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:34:59.746833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b14324-c596-42d9-b88b-7420642a5484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:34:59.748678 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c1b14324-c596-42d9-b88b-7420642a5484 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:34:59.828167 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b14324-c596-42d9-b88b-7420642a5484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:34:59.829617 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b14324-c596-42d9-b88b-7420642a5484 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:34:59.847965 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1398/2804] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:34:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:35:00.379307 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518d0ff-02ca-44c4-b972-668832368fe5 service nova] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:35:00.388555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518d0ff-02ca-44c4-b972-668832368fe5 service nova] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:35:00.392505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee72cc9-70fb-4894-9f87-6b0a65787901 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:35:00.395323 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee72cc9-70fb-4894-9f87-6b0a65787901 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:35:00.395323 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee72cc9-70fb-4894-9f87-6b0a65787901 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:35:00.395323 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee72cc9-70fb-4894-9f87-6b0a65787901 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:35:00.395676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-8ee72cc9-70fb-4894-9f87-6b0a65787901 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:35:00.422863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application [None req-8ee72cc9-70fb-4894-9f87-6b0a65787901 service cinder] Could not find project: b9724c9a575141d48bbfbde685774622.: keystone.exception.ProjectNotFound: Could not find project: b9724c9a575141d48bbfbde685774622. May 10 20:35:00.422863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 20:35:00.422863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 20:35:00.422863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 20:35:00.422863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 20:35:00.422863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 20:35:00.422863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 20:35:00.422863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 20:35:00.422863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 20:35:00.422863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 20:35:00.422863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 20:35:00.422863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 20:35:00.422863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get May 10 20:35:00.422863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') May 10 20:35:00.422863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call May 10 20:35:00.425672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() May 10 20:35:00.425672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data May 10 20:35:00.425672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( May 10 20:35:00.425672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:35:00.425672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:35:00.425672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token May 10 20:35:00.425672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application token = self._validate_token(token_id) May 10 20:35:00.425672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:35:00.425672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:35:00.425672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:35:00.425672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:35:00.425672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:35:00.425672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:35:00.425672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:35:00.425672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:35:00.425672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:35:00.425672 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:35:00.427014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:35:00.427014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:35:00.427014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:35:00.427014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:35:00.427014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token May 10 20:35:00.427014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) May 10 20:35:00.427014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint May 10 20:35:00.427014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application self._validate_token_resources() May 10 20:35:00.427014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources May 10 20:35:00.427014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): May 10 20:35:00.427014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project May 10 20:35:00.427014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( May 10 20:35:00.427014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 20:35:00.427014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 20:35:00.427014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 20:35:00.427014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 20:35:00.427014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 20:35:00.428205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.get_or_create( May 10 20:35:00.428205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 20:35:00.428205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application with Lock( May 10 20:35:00.428205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 20:35:00.428205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._enter() May 10 20:35:00.428205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 20:35:00.428205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 20:35:00.428205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 20:35:00.428205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.creator() May 10 20:35:00.428205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 20:35:00.428205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application created_value = creator( May 10 20:35:00.428205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 20:35:00.428205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 20:35:00.428205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 20:35:00.428205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 20:35:00.428205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 20:35:00.428205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 20:35:00.429378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: b9724c9a575141d48bbfbde685774622. May 10 20:35:00.429378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: ERROR keystone.server.flask.application May 10 20:35:00.429378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1407/2805] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:35:00 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 46 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) May 10 20:35:01.180516 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96cdd4-882b-4b59-98aa-634ee9701b18 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:35:01.181047 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96cdd4-882b-4b59-98aa-634ee9701b18 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:35:01.181047 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96cdd4-882b-4b59-98aa-634ee9701b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:35:01.207535 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-4d96cdd4-882b-4b59-98aa-634ee9701b18 None None] MFA Rules not processed for user `04ce1bdd6a234177ba7745d19d6fd950`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:35:01.208723 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-4d96cdd4-882b-4b59-98aa-634ee9701b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:35:01.215503 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1399/2806] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:35:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:35:22.583127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee72cc9-70fb-4894-9f87-6b0a65787901 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:35:22.591596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee72cc9-70fb-4894-9f87-6b0a65787901 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:35:22.595842 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ce2ac-73d3-4b84-8017-3a81d27df361 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:35:22.596479 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ce2ac-73d3-4b84-8017-3a81d27df361 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:35:22.596796 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ce2ac-73d3-4b84-8017-3a81d27df361 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:35:22.597109 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ce2ac-73d3-4b84-8017-3a81d27df361 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:35:22.598906 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-3d0ce2ac-73d3-4b84-8017-3a81d27df361 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:35:22.667945 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ce2ac-73d3-4b84-8017-3a81d27df361 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:35:22.669390 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ce2ac-73d3-4b84-8017-3a81d27df361 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:35:22.678489 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1408/2807] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:35:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:35:52.408350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96cdd4-882b-4b59-98aa-634ee9701b18 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:35:52.420319 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96cdd4-882b-4b59-98aa-634ee9701b18 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:35:52.427403 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce738fb2-ac62-40bf-a6df-b2d9b7e0ddcc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022093dea10d46478b3016395c712d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:35:52.428061 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce738fb2-ac62-40bf-a6df-b2d9b7e0ddcc service placement] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:35:52.428346 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce738fb2-ac62-40bf-a6df-b2d9b7e0ddcc service placement] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:35:52.428624 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce738fb2-ac62-40bf-a6df-b2d9b7e0ddcc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:35:52.430436 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-ce738fb2-ac62-40bf-a6df-b2d9b7e0ddcc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:35:52.442560 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce738fb2-ac62-40bf-a6df-b2d9b7e0ddcc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:35:52.444142 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce738fb2-ac62-40bf-a6df-b2d9b7e0ddcc service placement] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:35:52.454504 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1400/2808] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:35:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:37:49.182668 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ce2ac-73d3-4b84-8017-3a81d27df361 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:37:49.192146 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ce2ac-73d3-4b84-8017-3a81d27df361 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:37:49.195763 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751225f-63df-441d-8c42-0bcee5f4f2d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:37:49.196398 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751225f-63df-441d-8c42-0bcee5f4f2d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:37:49.196684 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751225f-63df-441d-8c42-0bcee5f4f2d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:37:49.196962 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751225f-63df-441d-8c42-0bcee5f4f2d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:37:49.198739 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-9751225f-63df-441d-8c42-0bcee5f4f2d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:37:49.208582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751225f-63df-441d-8c42-0bcee5f4f2d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:37:49.210003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751225f-63df-441d-8c42-0bcee5f4f2d8 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:37:49.219960 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1409/2809] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:37:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:38:00.908479 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce738fb2-ac62-40bf-a6df-b2d9b7e0ddcc service placement] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:38:00.917613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce738fb2-ac62-40bf-a6df-b2d9b7e0ddcc service placement] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:38:00.921868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec2e39-f30f-46d2-8887-d4e65ffbd461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:38:00.922541 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec2e39-f30f-46d2-8887-d4e65ffbd461 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:38:00.922823 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec2e39-f30f-46d2-8887-d4e65ffbd461 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:38:00.923116 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec2e39-f30f-46d2-8887-d4e65ffbd461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:38:00.924850 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b4ec2e39-f30f-46d2-8887-d4e65ffbd461 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:38:00.938472 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec2e39-f30f-46d2-8887-d4e65ffbd461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:38:00.939893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec2e39-f30f-46d2-8887-d4e65ffbd461 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:38:00.950700 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1401/2810] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:38:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:39:01.140662 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751225f-63df-441d-8c42-0bcee5f4f2d8 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:39:01.155067 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751225f-63df-441d-8c42-0bcee5f4f2d8 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:39:01.161597 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0ba37-9fa7-4608-ab10-c6192aa4a9dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:39:01.162272 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0ba37-9fa7-4608-ab10-c6192aa4a9dd service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:39:01.165434 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0ba37-9fa7-4608-ab10-c6192aa4a9dd service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:39:01.165434 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0ba37-9fa7-4608-ab10-c6192aa4a9dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:39:01.173191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0ba37-9fa7-4608-ab10-c6192aa4a9dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:39:01.174949 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0ba37-9fa7-4608-ab10-c6192aa4a9dd service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:39:01.185774 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1410/2811] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:39:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:39:59.806648 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec2e39-f30f-46d2-8887-d4e65ffbd461 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:39:59.869660 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec2e39-f30f-46d2-8887-d4e65ffbd461 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:39:59.875243 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca1594-98a7-47fc-aa36-f887e6b2484b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:39:59.875879 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca1594-98a7-47fc-aa36-f887e6b2484b service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:39:59.876164 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca1594-98a7-47fc-aa36-f887e6b2484b service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:39:59.876437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca1594-98a7-47fc-aa36-f887e6b2484b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:39:59.884256 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca1594-98a7-47fc-aa36-f887e6b2484b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:39:59.885827 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca1594-98a7-47fc-aa36-f887e6b2484b service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:39:59.895056 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1402/2812] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:39:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:40:23.305114 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0ba37-9fa7-4608-ab10-c6192aa4a9dd service nova] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:40:23.361333 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0ba37-9fa7-4608-ab10-c6192aa4a9dd service nova] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:40:23.366948 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5897814-a1cf-418a-ac40-9959eb8b9c87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:40:23.367640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5897814-a1cf-418a-ac40-9959eb8b9c87 service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:40:23.367956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5897814-a1cf-418a-ac40-9959eb8b9c87 service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:40:23.368235 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5897814-a1cf-418a-ac40-9959eb8b9c87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:40:23.376352 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5897814-a1cf-418a-ac40-9959eb8b9c87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:40:23.377892 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5897814-a1cf-418a-ac40-9959eb8b9c87 service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:40:23.403952 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1411/2813] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:40:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:41:00.156872 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca1594-98a7-47fc-aa36-f887e6b2484b service nova] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:41:00.158277 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-cbca1594-98a7-47fc-aa36-f887e6b2484b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:41:00.192976 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca1594-98a7-47fc-aa36-f887e6b2484b service nova] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:41:00.196897 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda015d-9b8c-4a22-aff8-a3239d6cdd18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:41:00.197437 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda015d-9b8c-4a22-aff8-a3239d6cdd18 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:41:00.197717 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda015d-9b8c-4a22-aff8-a3239d6cdd18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:41:00.197988 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda015d-9b8c-4a22-aff8-a3239d6cdd18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:41:00.207050 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda015d-9b8c-4a22-aff8-a3239d6cdd18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:41:00.208490 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda015d-9b8c-4a22-aff8-a3239d6cdd18 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:41:00.217067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1403/2814] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:41:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:41:02.443989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5897814-a1cf-418a-ac40-9959eb8b9c87 service nova] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:41:02.445364 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-b5897814-a1cf-418a-ac40-9959eb8b9c87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:41:02.480816 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5897814-a1cf-418a-ac40-9959eb8b9c87 service nova] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:41:02.485864 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6c76d-e3b7-459d-8343-12b6b003acf6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022093dea10d46478b3016395c712d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:41:02.486537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6c76d-e3b7-459d-8343-12b6b003acf6 service placement] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:41:02.486826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6c76d-e3b7-459d-8343-12b6b003acf6 service placement] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:41:02.487107 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6c76d-e3b7-459d-8343-12b6b003acf6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:41:02.495593 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6c76d-e3b7-459d-8343-12b6b003acf6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:41:02.497182 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6c76d-e3b7-459d-8343-12b6b003acf6 service placement] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:41:02.506921 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1412/2815] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:41:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:42:54.139321 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda015d-9b8c-4a22-aff8-a3239d6cdd18 service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:42:54.157157 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda015d-9b8c-4a22-aff8-a3239d6cdd18 service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:42:54.160812 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189073e-04dd-42f2-81be-71be8968e8f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:42:54.161444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189073e-04dd-42f2-81be-71be8968e8f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:42:54.161725 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189073e-04dd-42f2-81be-71be8968e8f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:42:54.161998 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189073e-04dd-42f2-81be-71be8968e8f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:42:54.169215 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189073e-04dd-42f2-81be-71be8968e8f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:42:54.173727 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189073e-04dd-42f2-81be-71be8968e8f1 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:42:54.178802 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1404/2816] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:42:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:43:00.669755 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6c76d-e3b7-459d-8343-12b6b003acf6 service placement] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:43:00.679250 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6c76d-e3b7-459d-8343-12b6b003acf6 service placement] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:43:00.683346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68e6c0-0749-449b-90ce-904810590a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:43:00.683912 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68e6c0-0749-449b-90ce-904810590a3e service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:43:00.684191 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68e6c0-0749-449b-90ce-904810590a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:43:00.684469 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68e6c0-0749-449b-90ce-904810590a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:43:00.694752 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68e6c0-0749-449b-90ce-904810590a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:43:00.696227 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68e6c0-0749-449b-90ce-904810590a3e service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:43:00.707821 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1413/2817] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:43:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:45:02.102487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189073e-04dd-42f2-81be-71be8968e8f1 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:45:02.160304 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189073e-04dd-42f2-81be-71be8968e8f1 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:45:02.163961 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b1ea8-1f2d-4be4-ae67-79e4ffa336cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:45:02.164617 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b1ea8-1f2d-4be4-ae67-79e4ffa336cb service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:45:02.164936 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b1ea8-1f2d-4be4-ae67-79e4ffa336cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:45:02.165214 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b1ea8-1f2d-4be4-ae67-79e4ffa336cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:45:02.167145 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-bd4b1ea8-1f2d-4be4-ae67-79e4ffa336cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:45:02.246255 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4b1ea8-1f2d-4be4-ae67-79e4ffa336cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:45:02.247690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4b1ea8-1f2d-4be4-ae67-79e4ffa336cb service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:45:02.268953 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1405/2818] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:45:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:46:07.386094 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68e6c0-0749-449b-90ce-904810590a3e service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:46:07.450135 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68e6c0-0749-449b-90ce-904810590a3e service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:46:07.455111 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e80a18-5ad3-459b-8e2e-c802a49340c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022093dea10d46478b3016395c712d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:46:07.455785 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e80a18-5ad3-459b-8e2e-c802a49340c4 service placement] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:46:07.456070 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e80a18-5ad3-459b-8e2e-c802a49340c4 service placement] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:46:07.456355 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e80a18-5ad3-459b-8e2e-c802a49340c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:46:07.458288 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-80e80a18-5ad3-459b-8e2e-c802a49340c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:46:07.469755 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e80a18-5ad3-459b-8e2e-c802a49340c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:46:07.471354 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e80a18-5ad3-459b-8e2e-c802a49340c4 service placement] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:46:07.480964 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1414/2819] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:46:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:47:02.458528 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b1ea8-1f2d-4be4-ae67-79e4ffa336cb service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:47:02.467765 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b1ea8-1f2d-4be4-ae67-79e4ffa336cb service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:47:02.471366 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c5caa-301a-4a53-8ca4-4d279e403a70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:47:02.472000 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c5caa-301a-4a53-8ca4-4d279e403a70 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:47:02.472286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c5caa-301a-4a53-8ca4-4d279e403a70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:47:02.472564 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c5caa-301a-4a53-8ca4-4d279e403a70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:47:02.474462 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c21c5caa-301a-4a53-8ca4-4d279e403a70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:47:02.543359 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c5caa-301a-4a53-8ca4-4d279e403a70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:47:02.544771 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c5caa-301a-4a53-8ca4-4d279e403a70 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:47:02.553451 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1406/2820] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:47:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:47:02.634326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e80a18-5ad3-459b-8e2e-c802a49340c4 service placement] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:47:02.644080 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e80a18-5ad3-459b-8e2e-c802a49340c4 service placement] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:47:02.647617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5be4fb-4cb3-4358-9218-72a5589a0400 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:47:02.648241 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5be4fb-4cb3-4358-9218-72a5589a0400 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:47:02.648523 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5be4fb-4cb3-4358-9218-72a5589a0400 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:47:02.648800 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5be4fb-4cb3-4358-9218-72a5589a0400 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:47:02.651249 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-eb5be4fb-4cb3-4358-9218-72a5589a0400 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:47:02.727117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5be4fb-4cb3-4358-9218-72a5589a0400 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:47:02.728532 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5be4fb-4cb3-4358-9218-72a5589a0400 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:47:02.745815 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1415/2821] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:47:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:50:02.391460 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c5caa-301a-4a53-8ca4-4d279e403a70 service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:50:02.393004 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-c21c5caa-301a-4a53-8ca4-4d279e403a70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:50:02.424808 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c5caa-301a-4a53-8ca4-4d279e403a70 service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:50:02.428499 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12fb5a-78b0-4ea0-88b9-62ec19afe9af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:50:02.429148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12fb5a-78b0-4ea0-88b9-62ec19afe9af service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:50:02.429432 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12fb5a-78b0-4ea0-88b9-62ec19afe9af service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:50:02.429711 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12fb5a-78b0-4ea0-88b9-62ec19afe9af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:50:02.438909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12fb5a-78b0-4ea0-88b9-62ec19afe9af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:50:02.440348 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12fb5a-78b0-4ea0-88b9-62ec19afe9af service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:50:02.449902 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1407/2822] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:50:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:51:03.613919 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5be4fb-4cb3-4358-9218-72a5589a0400 service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:51:03.615604 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-eb5be4fb-4cb3-4358-9218-72a5589a0400 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:51:03.715712 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5be4fb-4cb3-4358-9218-72a5589a0400 service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:51:03.720023 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ea574-9b94-4d73-aa50-61cc880a615a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:51:03.720671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ea574-9b94-4d73-aa50-61cc880a615a service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:51:03.720956 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ea574-9b94-4d73-aa50-61cc880a615a service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:51:03.721234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ea574-9b94-4d73-aa50-61cc880a615a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:51:03.723086 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-758ea574-9b94-4d73-aa50-61cc880a615a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:51:03.829120 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ea574-9b94-4d73-aa50-61cc880a615a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:51:03.830610 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ea574-9b94-4d73-aa50-61cc880a615a service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:51:03.849782 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1416/2823] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:51:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:51:11.242107 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12fb5a-78b0-4ea0-88b9-62ec19afe9af service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:51:11.243667 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-da12fb5a-78b0-4ea0-88b9-62ec19afe9af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:51:11.274593 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12fb5a-78b0-4ea0-88b9-62ec19afe9af service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:51:11.279974 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd3105-32f8-4188-887c-54b1341f01d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:51:11.280833 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd3105-32f8-4188-887c-54b1341f01d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:51:11.281116 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd3105-32f8-4188-887c-54b1341f01d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:51:11.281387 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd3105-32f8-4188-887c-54b1341f01d6 admin admin] PATH_INFO: `/v3/users/46916c5b15254c529bc52ecb711fdd76` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:51:11.282874 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd3105-32f8-4188-887c-54b1341f01d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46916c5b15254c529bc52ecb711fdd76)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:51:11.284776 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd3105-32f8-4188-887c-54b1341f01d6 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:51:11.335320 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ea574-9b94-4d73-aa50-61cc880a615a service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:51:11.336950 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-758ea574-9b94-4d73-aa50-61cc880a615a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:51:11.374007 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ea574-9b94-4d73-aa50-61cc880a615a service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:51:11.379017 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a33058-f554-4f74-a36a-eee58e2834fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022093dea10d46478b3016395c712d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:51:11.379652 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a33058-f554-4f74-a36a-eee58e2834fd service placement] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:51:11.379934 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a33058-f554-4f74-a36a-eee58e2834fd service placement] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:51:11.380209 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a33058-f554-4f74-a36a-eee58e2834fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:51:11.389474 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a33058-f554-4f74-a36a-eee58e2834fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:51:11.391055 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a33058-f554-4f74-a36a-eee58e2834fd service placement] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:51:11.402440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1417/2824] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:51:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:51:11.471127 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-6cfd3105-32f8-4188-887c-54b1341f01d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46916c5b15254c529bc52ecb711fdd76'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:11.471828 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-6cfd3105-32f8-4188-887c-54b1341f01d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46916c5b15254c529bc52ecb711fdd76'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:11.484925 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-6cfd3105-32f8-4188-887c-54b1341f01d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46916c5b15254c529bc52ecb711fdd76'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:11.487347 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-6cfd3105-32f8-4188-887c-54b1341f01d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46916c5b15254c529bc52ecb711fdd76'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:11.495555 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-6cfd3105-32f8-4188-887c-54b1341f01d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46916c5b15254c529bc52ecb711fdd76'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:11.614869 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1408/2825] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:51:11 2022] DELETE /identity/v3/users/46916c5b15254c529bc52ecb711fdd76 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:51:11.871412 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a33058-f554-4f74-a36a-eee58e2834fd service placement] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:51:11.885146 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a33058-f554-4f74-a36a-eee58e2834fd service placement] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:51:11.941614 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e3b0d-7532-415c-ab69-d46e54d38e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:51:11.942516 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e3b0d-7532-415c-ab69-d46e54d38e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:51:11.942807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e3b0d-7532-415c-ab69-d46e54d38e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:51:11.943085 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e3b0d-7532-415c-ab69-d46e54d38e72 admin admin] PATH_INFO: `/v3/projects/93dce3ce114d427895f446e0ed0f7e4f` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:51:11.954391 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e3b0d-7532-415c-ab69-d46e54d38e72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93dce3ce114d427895f446e0ed0f7e4f)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:51:11.956309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e3b0d-7532-415c-ab69-d46e54d38e72 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:51:12.051703 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-576e3b0d-7532-415c-ab69-d46e54d38e72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93dce3ce114d427895f446e0ed0f7e4f', 'request_id': 'req-576e3b0d-7532-415c-ab69-d46e54d38e72'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:12.120149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-576e3b0d-7532-415c-ab69-d46e54d38e72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93dce3ce114d427895f446e0ed0f7e4f', 'request_id': 'req-576e3b0d-7532-415c-ab69-d46e54d38e72'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:12.125807 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-576e3b0d-7532-415c-ab69-d46e54d38e72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93dce3ce114d427895f446e0ed0f7e4f', 'request_id': 'req-576e3b0d-7532-415c-ab69-d46e54d38e72'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:12.137110 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-576e3b0d-7532-415c-ab69-d46e54d38e72 admin admin] The token cache is being invalidate because project 93dce3ce114d427895f446e0ed0f7e4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:51:12.140605 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-576e3b0d-7532-415c-ab69-d46e54d38e72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:12.140605 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-576e3b0d-7532-415c-ab69-d46e54d38e72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:12.140605 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1418/2826] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:51:11 2022] DELETE /identity/v3/projects/93dce3ce114d427895f446e0ed0f7e4f => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:51:12.143604 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd3105-32f8-4188-887c-54b1341f01d6 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:51:12.144998 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-6cfd3105-32f8-4188-887c-54b1341f01d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:51:12.158764 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd3105-32f8-4188-887c-54b1341f01d6 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:51:12.212541 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cda94-00c8-43dd-80c0-eb0f783107f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:51:12.213402 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cda94-00c8-43dd-80c0-eb0f783107f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:51:12.213688 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cda94-00c8-43dd-80c0-eb0f783107f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:51:12.213965 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cda94-00c8-43dd-80c0-eb0f783107f9 admin admin] PATH_INFO: `/v3/users/5283855991dd4d5390e33f57312c9531` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:51:12.227346 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cda94-00c8-43dd-80c0-eb0f783107f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5283855991dd4d5390e33f57312c9531)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:51:12.229276 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cda94-00c8-43dd-80c0-eb0f783107f9 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:51:12.390314 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-d88cda94-00c8-43dd-80c0-eb0f783107f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5283855991dd4d5390e33f57312c9531'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:12.390957 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-d88cda94-00c8-43dd-80c0-eb0f783107f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5283855991dd4d5390e33f57312c9531'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:12.402496 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-d88cda94-00c8-43dd-80c0-eb0f783107f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5283855991dd4d5390e33f57312c9531'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:12.403127 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-d88cda94-00c8-43dd-80c0-eb0f783107f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5283855991dd4d5390e33f57312c9531'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:12.410876 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-d88cda94-00c8-43dd-80c0-eb0f783107f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5283855991dd4d5390e33f57312c9531'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:12.492328 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1409/2827] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:51:12 2022] DELETE /identity/v3/users/5283855991dd4d5390e33f57312c9531 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:51:12.715442 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e3b0d-7532-415c-ab69-d46e54d38e72 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:51:12.717518 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-576e3b0d-7532-415c-ab69-d46e54d38e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:51:12.730309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e3b0d-7532-415c-ab69-d46e54d38e72 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:51:12.787554 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75688863-0743-4572-a41b-c83bbbd50fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:51:12.788409 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75688863-0743-4572-a41b-c83bbbd50fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:51:12.788821 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75688863-0743-4572-a41b-c83bbbd50fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:51:12.788965 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75688863-0743-4572-a41b-c83bbbd50fde admin admin] PATH_INFO: `/v3/projects/673c4c7cfb9d4a98a79cd5f3e3979b08` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:51:12.799619 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75688863-0743-4572-a41b-c83bbbd50fde admin admin] RBAC: Authorizing `identity:delete_project(project_id=673c4c7cfb9d4a98a79cd5f3e3979b08)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:51:12.801525 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75688863-0743-4572-a41b-c83bbbd50fde admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:51:12.914674 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-75688863-0743-4572-a41b-c83bbbd50fde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '673c4c7cfb9d4a98a79cd5f3e3979b08', 'request_id': 'req-75688863-0743-4572-a41b-c83bbbd50fde'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:12.988036 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-75688863-0743-4572-a41b-c83bbbd50fde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '673c4c7cfb9d4a98a79cd5f3e3979b08', 'request_id': 'req-75688863-0743-4572-a41b-c83bbbd50fde'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:12.993418 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-75688863-0743-4572-a41b-c83bbbd50fde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '673c4c7cfb9d4a98a79cd5f3e3979b08', 'request_id': 'req-75688863-0743-4572-a41b-c83bbbd50fde'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:13.004979 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-75688863-0743-4572-a41b-c83bbbd50fde admin admin] The token cache is being invalidate because project 673c4c7cfb9d4a98a79cd5f3e3979b08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:51:13.005263 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-75688863-0743-4572-a41b-c83bbbd50fde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:13.005875 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-75688863-0743-4572-a41b-c83bbbd50fde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:51:13.007425 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1419/2828] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:51:12 2022] DELETE /identity/v3/projects/673c4c7cfb9d4a98a79cd5f3e3979b08 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:52:03.063581 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cda94-00c8-43dd-80c0-eb0f783107f9 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:52:03.064485 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-d88cda94-00c8-43dd-80c0-eb0f783107f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:52:03.129778 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cda94-00c8-43dd-80c0-eb0f783107f9 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:52:03.177726 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a9701-5b2c-4550-9d46-38d7a3cbae3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:52:03.178339 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a9701-5b2c-4550-9d46-38d7a3cbae3f service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:52:03.178623 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a9701-5b2c-4550-9d46-38d7a3cbae3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:52:03.178898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a9701-5b2c-4550-9d46-38d7a3cbae3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:52:03.180648 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-999a9701-5b2c-4550-9d46-38d7a3cbae3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:52:03.236486 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a9701-5b2c-4550-9d46-38d7a3cbae3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:52:03.237914 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a9701-5b2c-4550-9d46-38d7a3cbae3f service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:52:03.246601 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1410/2829] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:52:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:55:02.009157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75688863-0743-4572-a41b-c83bbbd50fde admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:55:02.010582 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-75688863-0743-4572-a41b-c83bbbd50fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:55:02.023152 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75688863-0743-4572-a41b-c83bbbd50fde admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:55:02.026979 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1edbc-5ddd-45e9-9f89-877561f4aa1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:55:02.027617 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1edbc-5ddd-45e9-9f89-877561f4aa1b service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:55:02.027904 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1edbc-5ddd-45e9-9f89-877561f4aa1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:55:02.028181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1edbc-5ddd-45e9-9f89-877561f4aa1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:55:02.030498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-5ff1edbc-5ddd-45e9-9f89-877561f4aa1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:55:02.086702 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1edbc-5ddd-45e9-9f89-877561f4aa1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:55:02.088127 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1edbc-5ddd-45e9-9f89-877561f4aa1b service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:55:02.096898 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1420/2830] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:55:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:56:00.477492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a9701-5b2c-4550-9d46-38d7a3cbae3f service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:56:00.478984 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-999a9701-5b2c-4550-9d46-38d7a3cbae3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:56:00.543391 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a9701-5b2c-4550-9d46-38d7a3cbae3f service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:56:00.547669 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902d5c0-a34b-4c69-820c-c55d77e4fce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:56:00.548305 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902d5c0-a34b-4c69-820c-c55d77e4fce2 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:56:00.548591 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902d5c0-a34b-4c69-820c-c55d77e4fce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:56:00.548870 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902d5c0-a34b-4c69-820c-c55d77e4fce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:56:00.550654 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-b902d5c0-a34b-4c69-820c-c55d77e4fce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:56:00.611227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902d5c0-a34b-4c69-820c-c55d77e4fce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:56:00.612649 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902d5c0-a34b-4c69-820c-c55d77e4fce2 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:56:00.621992 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1411/2831] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:56:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:56:01.225039 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1edbc-5ddd-45e9-9f89-877561f4aa1b service cinder] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:56:01.226666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-5ff1edbc-5ddd-45e9-9f89-877561f4aa1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:56:01.240350 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1edbc-5ddd-45e9-9f89-877561f4aa1b service cinder] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:56:01.245032 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536f54a-897a-4f1e-98d6-678c7562088b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:56:01.245809 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536f54a-897a-4f1e-98d6-678c7562088b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:56:01.246138 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536f54a-897a-4f1e-98d6-678c7562088b admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:56:01.246385 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536f54a-897a-4f1e-98d6-678c7562088b admin admin] PATH_INFO: `/v3/users/77e46909519247119a6bc011b6bcfc7c` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:56:01.260488 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536f54a-897a-4f1e-98d6-678c7562088b admin admin] RBAC: Authorizing `identity:delete_user(user_id=77e46909519247119a6bc011b6bcfc7c)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:56:01.262428 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536f54a-897a-4f1e-98d6-678c7562088b admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:56:01.441602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7536f54a-897a-4f1e-98d6-678c7562088b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77e46909519247119a6bc011b6bcfc7c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:56:01.443275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7536f54a-897a-4f1e-98d6-678c7562088b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77e46909519247119a6bc011b6bcfc7c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:56:01.443275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7536f54a-897a-4f1e-98d6-678c7562088b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77e46909519247119a6bc011b6bcfc7c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:56:01.455554 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7536f54a-897a-4f1e-98d6-678c7562088b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77e46909519247119a6bc011b6bcfc7c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:56:01.462049 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-7536f54a-897a-4f1e-98d6-678c7562088b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77e46909519247119a6bc011b6bcfc7c'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:56:01.565057 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1421/2832] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:56:01 2022] DELETE /identity/v3/users/77e46909519247119a6bc011b6bcfc7c => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:56:01.818632 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902d5c0-a34b-4c69-820c-c55d77e4fce2 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:56:01.828625 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902d5c0-a34b-4c69-820c-c55d77e4fce2 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:56:01.884090 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df07034-1792-4058-a51e-76500ba56951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:56:01.884943 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df07034-1792-4058-a51e-76500ba56951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:56:01.885235 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df07034-1792-4058-a51e-76500ba56951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:56:01.885490 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df07034-1792-4058-a51e-76500ba56951 admin admin] PATH_INFO: `/v3/projects/498ce5260ee44f5d9b1ebc74fecedfbf` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:56:01.896599 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df07034-1792-4058-a51e-76500ba56951 admin admin] RBAC: Authorizing `identity:delete_project(project_id=498ce5260ee44f5d9b1ebc74fecedfbf)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:56:01.900045 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df07034-1792-4058-a51e-76500ba56951 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:56:02.005508 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-6df07034-1792-4058-a51e-76500ba56951 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '498ce5260ee44f5d9b1ebc74fecedfbf', 'request_id': 'req-6df07034-1792-4058-a51e-76500ba56951'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:56:02.079151 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-6df07034-1792-4058-a51e-76500ba56951 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '498ce5260ee44f5d9b1ebc74fecedfbf', 'request_id': 'req-6df07034-1792-4058-a51e-76500ba56951'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:56:02.084570 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-6df07034-1792-4058-a51e-76500ba56951 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '498ce5260ee44f5d9b1ebc74fecedfbf', 'request_id': 'req-6df07034-1792-4058-a51e-76500ba56951'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:56:02.097781 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-6df07034-1792-4058-a51e-76500ba56951 admin admin] The token cache is being invalidate because project 498ce5260ee44f5d9b1ebc74fecedfbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:56:02.098064 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-6df07034-1792-4058-a51e-76500ba56951 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:56:02.098715 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-6df07034-1792-4058-a51e-76500ba56951 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:56:02.100227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1412/2833] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:56:01 2022] DELETE /identity/v3/projects/498ce5260ee44f5d9b1ebc74fecedfbf => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:56:15.337897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536f54a-897a-4f1e-98d6-678c7562088b admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:56:15.339300 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7536f54a-897a-4f1e-98d6-678c7562088b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:56:15.407171 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536f54a-897a-4f1e-98d6-678c7562088b admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:56:15.463653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b104c2-bef7-40cc-bd5b-776c53a46c40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022093dea10d46478b3016395c712d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:56:15.464302 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b104c2-bef7-40cc-bd5b-776c53a46c40 service placement] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:56:15.464589 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b104c2-bef7-40cc-bd5b-776c53a46c40 service placement] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:56:15.464868 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b104c2-bef7-40cc-bd5b-776c53a46c40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:56:15.466666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-55b104c2-bef7-40cc-bd5b-776c53a46c40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:56:15.478300 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b104c2-bef7-40cc-bd5b-776c53a46c40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:56:15.479879 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b104c2-bef7-40cc-bd5b-776c53a46c40 service placement] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:56:15.489739 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1422/2834] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:56:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:57:02.764275 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df07034-1792-4058-a51e-76500ba56951 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:02.765656 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-6df07034-1792-4058-a51e-76500ba56951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:02.866851 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df07034-1792-4058-a51e-76500ba56951 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:02.872600 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26633d-2b8e-4f3f-a45c-f815271c2565 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:02.873240 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26633d-2b8e-4f3f-a45c-f815271c2565 service nova] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:02.873524 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26633d-2b8e-4f3f-a45c-f815271c2565 service nova] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:02.873822 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26633d-2b8e-4f3f-a45c-f815271c2565 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:02.875623 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-ae26633d-2b8e-4f3f-a45c-f815271c2565 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:02.950860 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26633d-2b8e-4f3f-a45c-f815271c2565 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:02.952487 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26633d-2b8e-4f3f-a45c-f815271c2565 service nova] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:02.974899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1413/2835] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:57:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:57:03.024176 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b104c2-bef7-40cc-bd5b-776c53a46c40 service placement] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:03.025550 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-55b104c2-bef7-40cc-bd5b-776c53a46c40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:03.082357 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b104c2-bef7-40cc-bd5b-776c53a46c40 service placement] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:03.086030 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304e661-d9ca-4e38-9d43-73dc0848e326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:03.086692 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304e661-d9ca-4e38-9d43-73dc0848e326 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:03.086979 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304e661-d9ca-4e38-9d43-73dc0848e326 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:03.087250 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304e661-d9ca-4e38-9d43-73dc0848e326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:03.088964 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-e304e661-d9ca-4e38-9d43-73dc0848e326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:03.154234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304e661-d9ca-4e38-9d43-73dc0848e326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:03.158723 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304e661-d9ca-4e38-9d43-73dc0848e326 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:03.168280 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1423/2836] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:57:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:57:06.873566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26633d-2b8e-4f3f-a45c-f815271c2565 service nova] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:06.875805 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-ae26633d-2b8e-4f3f-a45c-f815271c2565 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:06.890252 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26633d-2b8e-4f3f-a45c-f815271c2565 service nova] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:06.894901 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82320f-2e56-44ee-aec0-e808d9733673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:06.896027 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82320f-2e56-44ee-aec0-e808d9733673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:06.896312 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82320f-2e56-44ee-aec0-e808d9733673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:06.896589 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82320f-2e56-44ee-aec0-e808d9733673 admin admin] PATH_INFO: `/v3/users/04ce1bdd6a234177ba7745d19d6fd950` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:06.897838 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82320f-2e56-44ee-aec0-e808d9733673 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04ce1bdd6a234177ba7745d19d6fd950)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:06.899788 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82320f-2e56-44ee-aec0-e808d9733673 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:07.068959 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0a82320f-2e56-44ee-aec0-e808d9733673 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04ce1bdd6a234177ba7745d19d6fd950'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:07.069608 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0a82320f-2e56-44ee-aec0-e808d9733673 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04ce1bdd6a234177ba7745d19d6fd950'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:07.081286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0a82320f-2e56-44ee-aec0-e808d9733673 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04ce1bdd6a234177ba7745d19d6fd950'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:07.083206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0a82320f-2e56-44ee-aec0-e808d9733673 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04ce1bdd6a234177ba7745d19d6fd950'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:07.088698 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-0a82320f-2e56-44ee-aec0-e808d9733673 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04ce1bdd6a234177ba7745d19d6fd950'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:07.158617 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1414/2837] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:57:06 2022] DELETE /identity/v3/users/04ce1bdd6a234177ba7745d19d6fd950 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:07.371972 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304e661-d9ca-4e38-9d43-73dc0848e326 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:07.382987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304e661-d9ca-4e38-9d43-73dc0848e326 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:07.444194 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c2dde-c19d-4a9e-98c0-09f231054f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:07.445050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c2dde-c19d-4a9e-98c0-09f231054f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:07.445327 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c2dde-c19d-4a9e-98c0-09f231054f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:07.445604 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c2dde-c19d-4a9e-98c0-09f231054f15 admin admin] PATH_INFO: `/v3/projects/1d1c68d15d634e81acba3ca215f5548c` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:07.446795 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c2dde-c19d-4a9e-98c0-09f231054f15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d1c68d15d634e81acba3ca215f5548c)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:07.448702 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c2dde-c19d-4a9e-98c0-09f231054f15 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:07.540238 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-987c2dde-c19d-4a9e-98c0-09f231054f15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d1c68d15d634e81acba3ca215f5548c', 'request_id': 'req-987c2dde-c19d-4a9e-98c0-09f231054f15'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:07.586523 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-987c2dde-c19d-4a9e-98c0-09f231054f15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d1c68d15d634e81acba3ca215f5548c', 'request_id': 'req-987c2dde-c19d-4a9e-98c0-09f231054f15'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:07.592100 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-987c2dde-c19d-4a9e-98c0-09f231054f15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d1c68d15d634e81acba3ca215f5548c', 'request_id': 'req-987c2dde-c19d-4a9e-98c0-09f231054f15'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:07.603527 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-987c2dde-c19d-4a9e-98c0-09f231054f15 admin admin] The token cache is being invalidate because project 1d1c68d15d634e81acba3ca215f5548c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:57:07.603812 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-987c2dde-c19d-4a9e-98c0-09f231054f15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:07.604443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-987c2dde-c19d-4a9e-98c0-09f231054f15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:07.606016 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1424/2838] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:57:07 2022] DELETE /identity/v3/projects/1d1c68d15d634e81acba3ca215f5548c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:07.611798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbc68a-7c60-4856-b327-3c0ef3af18b1 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:07.612209 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbc68a-7c60-4856-b327-3c0ef3af18b1 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:07.612554 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbc68a-7c60-4856-b327-3c0ef3af18b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:07.670559 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-45fbc68a-7c60-4856-b327-3c0ef3af18b1 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:07.671832 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-45fbc68a-7c60-4856-b327-3c0ef3af18b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:07.729890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1415/2839] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:57:07.738483 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2108e-117d-49bd-a607-ad75aed9dd75 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:07.738794 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2108e-117d-49bd-a607-ad75aed9dd75 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:07.739056 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2108e-117d-49bd-a607-ad75aed9dd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:07.757664 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-03c2108e-117d-49bd-a607-ad75aed9dd75 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:07.758895 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-03c2108e-117d-49bd-a607-ad75aed9dd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:07.767193 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1425/2840] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:57:07.771668 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbc68a-7c60-4856-b327-3c0ef3af18b1 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:07.781706 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbc68a-7c60-4856-b327-3c0ef3af18b1 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:07.786184 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4334d-db29-4337-a9fc-a3f071712be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:07.786823 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4334d-db29-4337-a9fc-a3f071712be0 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:07.787103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4334d-db29-4337-a9fc-a3f071712be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:07.787372 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4334d-db29-4337-a9fc-a3f071712be0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:07.788087 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4334d-db29-4337-a9fc-a3f071712be0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:57:07.788387 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4334d-db29-4337-a9fc-a3f071712be0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:07.789950 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4334d-db29-4337-a9fc-a3f071712be0 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:07.801229 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1416/2841] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:57:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:57:07.805212 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2108e-117d-49bd-a607-ad75aed9dd75 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:07.814584 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2108e-117d-49bd-a607-ad75aed9dd75 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:07.819232 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee8ce7-86ef-40c1-8159-8aa60a997d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:07.819901 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee8ce7-86ef-40c1-8159-8aa60a997d58 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:07.820187 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee8ce7-86ef-40c1-8159-8aa60a997d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:07.820468 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee8ce7-86ef-40c1-8159-8aa60a997d58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:07.821178 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee8ce7-86ef-40c1-8159-8aa60a997d58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:07.823088 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee8ce7-86ef-40c1-8159-8aa60a997d58 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:07.925207 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1426/2842] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:57:07 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:57:07.929067 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4334d-db29-4337-a9fc-a3f071712be0 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:07.939529 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4334d-db29-4337-a9fc-a3f071712be0 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:07.993346 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89dfb2-ca44-4d37-a800-7aed0d8b66b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:07.994013 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89dfb2-ca44-4d37-a800-7aed0d8b66b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:07.994331 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89dfb2-ca44-4d37-a800-7aed0d8b66b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:07.994611 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89dfb2-ca44-4d37-a800-7aed0d8b66b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:07.995302 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b89dfb2-ca44-4d37-a800-7aed0d8b66b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:07.997206 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b89dfb2-ca44-4d37-a800-7aed0d8b66b6 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:08.067193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1417/2843] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:57:07 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:57:08.071212 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee8ce7-86ef-40c1-8159-8aa60a997d58 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:08.082104 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee8ce7-86ef-40c1-8159-8aa60a997d58 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:08.086762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34658e-e4c2-419a-9323-844f2174e84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:08.089858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34658e-e4c2-419a-9323-844f2174e84c admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:08.089858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34658e-e4c2-419a-9323-844f2174e84c admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:08.089858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34658e-e4c2-419a-9323-844f2174e84c admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:08.089858 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34658e-e4c2-419a-9323-844f2174e84c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:08.090287 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34658e-e4c2-419a-9323-844f2174e84c admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:08.103248 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1427/2844] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:57:08.104541 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89dfb2-ca44-4d37-a800-7aed0d8b66b6 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:08.114580 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89dfb2-ca44-4d37-a800-7aed0d8b66b6 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:08.119182 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c8755-55fc-47e8-afff-32ee211823d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:08.119825 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c8755-55fc-47e8-afff-32ee211823d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:08.120110 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c8755-55fc-47e8-afff-32ee211823d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:08.120383 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c8755-55fc-47e8-afff-32ee211823d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:08.121107 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c8755-55fc-47e8-afff-32ee211823d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:08.122819 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c8755-55fc-47e8-afff-32ee211823d4 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:08.133172 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1418/2845] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:57:08.137165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34658e-e4c2-419a-9323-844f2174e84c admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:08.148280 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34658e-e4c2-419a-9323-844f2174e84c admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:08.153433 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3796315-44fc-4959-9e77-ae222dde715b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:08.154375 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3796315-44fc-4959-9e77-ae222dde715b admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:08.154665 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3796315-44fc-4959-9e77-ae222dde715b admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:08.154944 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3796315-44fc-4959-9e77-ae222dde715b admin admin] PATH_INFO: `/v3/projects/72ba145b02734605a3dfee6533c82fe3/users/628e2548877349f78ce1cd7cb0a1e2b5/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:08.169425 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3796315-44fc-4959-9e77-ae222dde715b admin admin] RBAC: Authorizing `identity:create_grant(project_id=72ba145b02734605a3dfee6533c82fe3, user_id=628e2548877349f78ce1cd7cb0a1e2b5, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:08.172182 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3796315-44fc-4959-9e77-ae222dde715b admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:08.244125 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1428/2846] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:57:08 2022] PUT /identity/v3/projects/72ba145b02734605a3dfee6533c82fe3/users/628e2548877349f78ce1cd7cb0a1e2b5/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:08.254713 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282cc19-845c-4b1d-af5e-00ffc33e4ff1 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:08.255001 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282cc19-845c-4b1d-af5e-00ffc33e4ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:08.255279 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282cc19-845c-4b1d-af5e-00ffc33e4ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:08.282934 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-6282cc19-845c-4b1d-af5e-00ffc33e4ff1 None None] MFA Rules not processed for user `628e2548877349f78ce1cd7cb0a1e2b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:08.284183 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-6282cc19-845c-4b1d-af5e-00ffc33e4ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:08.347344 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1419/2847] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:57:08.359085 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3796315-44fc-4959-9e77-ae222dde715b admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:08.360799 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-e3796315-44fc-4959-9e77-ae222dde715b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:08.432064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3796315-44fc-4959-9e77-ae222dde715b admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:08.438197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a63df-4984-4714-bcd1-f0130d277176 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae922b2da4814d10bed1fa345700ac04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:08.438854 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a63df-4984-4714-bcd1-f0130d277176 service nova] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:08.439145 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a63df-4984-4714-bcd1-f0130d277176 service nova] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:08.439425 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a63df-4984-4714-bcd1-f0130d277176 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:08.447391 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a63df-4984-4714-bcd1-f0130d277176 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:08.448987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a63df-4984-4714-bcd1-f0130d277176 service nova] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:08.457735 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1429/2848] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:57:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:57:08.477613 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282cc19-845c-4b1d-af5e-00ffc33e4ff1 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:08.478977 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-6282cc19-845c-4b1d-af5e-00ffc33e4ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:08.534557 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282cc19-845c-4b1d-af5e-00ffc33e4ff1 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:08.538111 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c38d3-180a-4e10-ae01-a16b68e22554 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:08.538767 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c38d3-180a-4e10-ae01-a16b68e22554 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:08.539049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c38d3-180a-4e10-ae01-a16b68e22554 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:08.539323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c38d3-180a-4e10-ae01-a16b68e22554 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:08.541068 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-122c38d3-180a-4e10-ae01-a16b68e22554 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:08.554404 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c38d3-180a-4e10-ae01-a16b68e22554 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:08.555818 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c38d3-180a-4e10-ae01-a16b68e22554 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:08.566956 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1420/2849] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:57:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:57:16.480129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a63df-4984-4714-bcd1-f0130d277176 service nova] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:16.540278 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a63df-4984-4714-bcd1-f0130d277176 service nova] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:16.544872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813afa9-a479-4ece-8f19-d21a024eadbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:16.545728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813afa9-a479-4ece-8f19-d21a024eadbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:16.546015 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813afa9-a479-4ece-8f19-d21a024eadbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:16.546324 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813afa9-a479-4ece-8f19-d21a024eadbc admin admin] PATH_INFO: `/v3/users/628e2548877349f78ce1cd7cb0a1e2b5` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:16.547547 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813afa9-a479-4ece-8f19-d21a024eadbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=628e2548877349f78ce1cd7cb0a1e2b5)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:16.550002 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813afa9-a479-4ece-8f19-d21a024eadbc admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:16.725185 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1813afa9-a479-4ece-8f19-d21a024eadbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '628e2548877349f78ce1cd7cb0a1e2b5'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:16.725854 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1813afa9-a479-4ece-8f19-d21a024eadbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '628e2548877349f78ce1cd7cb0a1e2b5'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:16.726474 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1813afa9-a479-4ece-8f19-d21a024eadbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '628e2548877349f78ce1cd7cb0a1e2b5'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:16.739020 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1813afa9-a479-4ece-8f19-d21a024eadbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '628e2548877349f78ce1cd7cb0a1e2b5'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:16.745821 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1813afa9-a479-4ece-8f19-d21a024eadbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '628e2548877349f78ce1cd7cb0a1e2b5'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:16.815287 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1430/2850] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:57:16 2022] DELETE /identity/v3/users/628e2548877349f78ce1cd7cb0a1e2b5 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:16.819771 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bc751-7ccd-4f5b-bec3-6e6634cdb493 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:16.820251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bc751-7ccd-4f5b-bec3-6e6634cdb493 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:16.820347 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bc751-7ccd-4f5b-bec3-6e6634cdb493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:16.838893 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-566bc751-7ccd-4f5b-bec3-6e6634cdb493 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:16.840087 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-566bc751-7ccd-4f5b-bec3-6e6634cdb493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:16.897465 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1421/2851] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:57:16.907615 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813afa9-a479-4ece-8f19-d21a024eadbc admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:16.909013 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-1813afa9-a479-4ece-8f19-d21a024eadbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:16.966049 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813afa9-a479-4ece-8f19-d21a024eadbc admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:16.970233 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfb234-af17-470b-8be1-05aec2a28939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:16.970873 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfb234-af17-470b-8be1-05aec2a28939 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:16.971155 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfb234-af17-470b-8be1-05aec2a28939 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:16.971429 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfb234-af17-470b-8be1-05aec2a28939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:16.980197 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bfb234-af17-470b-8be1-05aec2a28939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:16.981547 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bfb234-af17-470b-8be1-05aec2a28939 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:16.991569 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1431/2852] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:57:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 20:57:17.400282 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bc751-7ccd-4f5b-bec3-6e6634cdb493 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:17.405931 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-566bc751-7ccd-4f5b-bec3-6e6634cdb493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:17.414537 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bc751-7ccd-4f5b-bec3-6e6634cdb493 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:17.429151 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460acd64-11e2-419c-9d93-2a74df7e4aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:17.430075 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460acd64-11e2-419c-9d93-2a74df7e4aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:17.430350 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460acd64-11e2-419c-9d93-2a74df7e4aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:17.430637 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460acd64-11e2-419c-9d93-2a74df7e4aae admin admin] PATH_INFO: `/v3/projects/72ba145b02734605a3dfee6533c82fe3` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:17.431823 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460acd64-11e2-419c-9d93-2a74df7e4aae admin admin] RBAC: Authorizing `identity:delete_project(project_id=72ba145b02734605a3dfee6533c82fe3)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:17.433733 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460acd64-11e2-419c-9d93-2a74df7e4aae admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:17.563840 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-460acd64-11e2-419c-9d93-2a74df7e4aae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72ba145b02734605a3dfee6533c82fe3', 'request_id': 'req-460acd64-11e2-419c-9d93-2a74df7e4aae'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:17.643444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-460acd64-11e2-419c-9d93-2a74df7e4aae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72ba145b02734605a3dfee6533c82fe3', 'request_id': 'req-460acd64-11e2-419c-9d93-2a74df7e4aae'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:17.649293 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-460acd64-11e2-419c-9d93-2a74df7e4aae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72ba145b02734605a3dfee6533c82fe3', 'request_id': 'req-460acd64-11e2-419c-9d93-2a74df7e4aae'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:17.660182 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-460acd64-11e2-419c-9d93-2a74df7e4aae admin admin] The token cache is being invalidate because project 72ba145b02734605a3dfee6533c82fe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:57:17.660463 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-460acd64-11e2-419c-9d93-2a74df7e4aae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:17.661084 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-460acd64-11e2-419c-9d93-2a74df7e4aae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:17.662654 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1422/2853] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:57:17 2022] DELETE /identity/v3/projects/72ba145b02734605a3dfee6533c82fe3 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:17.667671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976992b-4727-4e63-8391-b2f1d0a7fad7 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:17.668146 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976992b-4727-4e63-8391-b2f1d0a7fad7 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:17.668243 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976992b-4727-4e63-8391-b2f1d0a7fad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:17.685552 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-c976992b-4727-4e63-8391-b2f1d0a7fad7 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:17.686903 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-c976992b-4727-4e63-8391-b2f1d0a7fad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:17.743388 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1432/2854] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:57:17.751612 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522b6d3-1870-43cb-849e-dfd87801c985 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:17.751899 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522b6d3-1870-43cb-849e-dfd87801c985 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:17.752178 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522b6d3-1870-43cb-849e-dfd87801c985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:17.770784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-f522b6d3-1870-43cb-849e-dfd87801c985 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:17.772002 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f522b6d3-1870-43cb-849e-dfd87801c985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:17.779778 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1423/2855] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:57:17.783757 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976992b-4727-4e63-8391-b2f1d0a7fad7 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:17.793234 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976992b-4727-4e63-8391-b2f1d0a7fad7 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:17.797765 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc3128-dd25-4bd4-9171-1fdfa761a44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:17.798440 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc3128-dd25-4bd4-9171-1fdfa761a44b admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:17.798728 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc3128-dd25-4bd4-9171-1fdfa761a44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:17.799006 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc3128-dd25-4bd4-9171-1fdfa761a44b admin admin] PATH_INFO: `/v3/domains` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:17.799736 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc3128-dd25-4bd4-9171-1fdfa761a44b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80515) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:57:17.800047 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc3128-dd25-4bd4-9171-1fdfa761a44b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:17.801653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc3128-dd25-4bd4-9171-1fdfa761a44b admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:17.812903 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1433/2856] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:57:17.817291 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522b6d3-1870-43cb-849e-dfd87801c985 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:17.826250 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522b6d3-1870-43cb-849e-dfd87801c985 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:17.831443 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82620d08-d582-44a1-be86-8c4ff33ac270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:17.832082 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82620d08-d582-44a1-be86-8c4ff33ac270 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:17.832362 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82620d08-d582-44a1-be86-8c4ff33ac270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:17.832634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82620d08-d582-44a1-be86-8c4ff33ac270 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:17.833328 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82620d08-d582-44a1-be86-8c4ff33ac270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:17.835227 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82620d08-d582-44a1-be86-8c4ff33ac270 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:17.919025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1424/2857] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:57:17 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:57:17.922878 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc3128-dd25-4bd4-9171-1fdfa761a44b admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:17.932941 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc3128-dd25-4bd4-9171-1fdfa761a44b admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:17.987609 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3c974-a073-47f8-a24d-a1c2e6d2fef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:17.988269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3c974-a073-47f8-a24d-a1c2e6d2fef9 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:17.988555 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3c974-a073-47f8-a24d-a1c2e6d2fef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:17.988831 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3c974-a073-47f8-a24d-a1c2e6d2fef9 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:17.989537 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3c974-a073-47f8-a24d-a1c2e6d2fef9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:17.991460 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3c974-a073-47f8-a24d-a1c2e6d2fef9 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:18.066676 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1434/2858] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:57:17 2022] POST /identity/v3/users => generated 283 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:57:18.070922 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82620d08-d582-44a1-be86-8c4ff33ac270 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:18.081212 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82620d08-d582-44a1-be86-8c4ff33ac270 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:18.085898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a38ef-eeca-4075-ae95-272bf2d87a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:18.086584 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a38ef-eeca-4075-ae95-272bf2d87a6d admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:18.086876 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a38ef-eeca-4075-ae95-272bf2d87a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:18.087155 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a38ef-eeca-4075-ae95-272bf2d87a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:18.087882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a38ef-eeca-4075-ae95-272bf2d87a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:18.089471 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a38ef-eeca-4075-ae95-272bf2d87a6d admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:18.099898 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1425/2859] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:57:18.103716 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3c974-a073-47f8-a24d-a1c2e6d2fef9 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:18.113869 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3c974-a073-47f8-a24d-a1c2e6d2fef9 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:18.118616 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ebe9c-4a53-4373-889d-1aca4df0a98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:18.119261 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ebe9c-4a53-4373-889d-1aca4df0a98e admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:18.119550 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ebe9c-4a53-4373-889d-1aca4df0a98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:18.119826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ebe9c-4a53-4373-889d-1aca4df0a98e admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:18.120556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ebe9c-4a53-4373-889d-1aca4df0a98e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:18.122153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ebe9c-4a53-4373-889d-1aca4df0a98e admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:18.132347 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1435/2860] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:57:18.136244 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a38ef-eeca-4075-ae95-272bf2d87a6d admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:18.146023 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a38ef-eeca-4075-ae95-272bf2d87a6d admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:18.150585 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304f0fb-b04e-4b33-bba3-7a6002b47133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:18.151444 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304f0fb-b04e-4b33-bba3-7a6002b47133 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:18.151725 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304f0fb-b04e-4b33-bba3-7a6002b47133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:18.152379 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304f0fb-b04e-4b33-bba3-7a6002b47133 admin admin] PATH_INFO: `/v3/projects/d4be45b752664f3c86a051d61e2fb947/users/38200536129a4c27b6f8734590744763/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:18.166364 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304f0fb-b04e-4b33-bba3-7a6002b47133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4be45b752664f3c86a051d61e2fb947, user_id=38200536129a4c27b6f8734590744763, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:18.168981 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304f0fb-b04e-4b33-bba3-7a6002b47133 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:18.249564 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1426/2861] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:57:18 2022] PUT /identity/v3/projects/d4be45b752664f3c86a051d61e2fb947/users/38200536129a4c27b6f8734590744763/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:57:18.306067 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253ef2a-05fe-4e5d-9d16-0412f0b8a706 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:18.306387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253ef2a-05fe-4e5d-9d16-0412f0b8a706 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:18.306667 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253ef2a-05fe-4e5d-9d16-0412f0b8a706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:18.332778 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-7253ef2a-05fe-4e5d-9d16-0412f0b8a706 None None] MFA Rules not processed for user `38200536129a4c27b6f8734590744763`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:18.333906 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7253ef2a-05fe-4e5d-9d16-0412f0b8a706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:18.394062 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1436/2862] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:57:18.398850 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304f0fb-b04e-4b33-bba3-7a6002b47133 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:18.458907 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304f0fb-b04e-4b33-bba3-7a6002b47133 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:18.463610 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf6f75-3b6e-4627-b39e-b1bc1e3bb680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:18.464262 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf6f75-3b6e-4627-b39e-b1bc1e3bb680 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:18.464546 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf6f75-3b6e-4627-b39e-b1bc1e3bb680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:18.464821 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf6f75-3b6e-4627-b39e-b1bc1e3bb680 admin admin] PATH_INFO: `/v3/projects` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:18.465555 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf6f75-3b6e-4627-b39e-b1bc1e3bb680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:18.467499 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf6f75-3b6e-4627-b39e-b1bc1e3bb680 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:18.548748 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1427/2863] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:57:18 2022] POST /identity/v3/projects => generated 365 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:57:18.553174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253ef2a-05fe-4e5d-9d16-0412f0b8a706 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:18.563917 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253ef2a-05fe-4e5d-9d16-0412f0b8a706 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:18.618280 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adba985-8474-4c07-a227-dfdaf27dd919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:18.618869 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adba985-8474-4c07-a227-dfdaf27dd919 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:18.619153 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adba985-8474-4c07-a227-dfdaf27dd919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:18.619427 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adba985-8474-4c07-a227-dfdaf27dd919 admin admin] PATH_INFO: `/v3/users` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:18.620126 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adba985-8474-4c07-a227-dfdaf27dd919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:18.622013 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adba985-8474-4c07-a227-dfdaf27dd919 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:18.714744 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1437/2864] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:57:18 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:57:18.721273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf6f75-3b6e-4627-b39e-b1bc1e3bb680 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:18.731840 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf6f75-3b6e-4627-b39e-b1bc1e3bb680 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:18.736508 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe414f7-c489-483f-b292-7834a7a7ea08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:18.737169 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe414f7-c489-483f-b292-7834a7a7ea08 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:18.737458 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe414f7-c489-483f-b292-7834a7a7ea08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:18.737864 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe414f7-c489-483f-b292-7834a7a7ea08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:18.738634 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe414f7-c489-483f-b292-7834a7a7ea08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:18.740261 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe414f7-c489-483f-b292-7834a7a7ea08 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:18.750758 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1428/2865] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:57:18.754921 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adba985-8474-4c07-a227-dfdaf27dd919 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:18.765100 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adba985-8474-4c07-a227-dfdaf27dd919 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:18.770630 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7be1a8-d451-4549-b504-d38a221f5917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:18.771519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7be1a8-d451-4549-b504-d38a221f5917 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:18.771806 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7be1a8-d451-4549-b504-d38a221f5917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:18.772088 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7be1a8-d451-4549-b504-d38a221f5917 admin admin] PATH_INFO: `/v3/projects/7b1f27f395004d3fb0d34f05c39ac357/users/0a19931bb6ad4e59ab61c7e36892d2e9/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:18.787394 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7be1a8-d451-4549-b504-d38a221f5917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1f27f395004d3fb0d34f05c39ac357, user_id=0a19931bb6ad4e59ab61c7e36892d2e9, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:18.790099 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7be1a8-d451-4549-b504-d38a221f5917 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:18.867205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1438/2866] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:57:18 2022] PUT /identity/v3/projects/7b1f27f395004d3fb0d34f05c39ac357/users/0a19931bb6ad4e59ab61c7e36892d2e9/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:57:18.872772 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe414f7-c489-483f-b292-7834a7a7ea08 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:18.882738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe414f7-c489-483f-b292-7834a7a7ea08 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:18.937597 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f9af4-8a4f-4d67-b3db-c40191fc3f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:18.938294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f9af4-8a4f-4d67-b3db-c40191fc3f9d admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:18.938577 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f9af4-8a4f-4d67-b3db-c40191fc3f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:18.938849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f9af4-8a4f-4d67-b3db-c40191fc3f9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:18.939570 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f9af4-8a4f-4d67-b3db-c40191fc3f9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:18.941148 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f9af4-8a4f-4d67-b3db-c40191fc3f9d admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:18.950795 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1429/2867] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:57:18.955762 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7be1a8-d451-4549-b504-d38a221f5917 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:18.966064 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7be1a8-d451-4549-b504-d38a221f5917 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:18.971229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdedb70-91c6-4c0f-aef1-ab1f8355ebed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:18.972094 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdedb70-91c6-4c0f-aef1-ab1f8355ebed admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:18.972376 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdedb70-91c6-4c0f-aef1-ab1f8355ebed admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:18.972653 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdedb70-91c6-4c0f-aef1-ab1f8355ebed admin admin] PATH_INFO: `/v3/projects/7b1f27f395004d3fb0d34f05c39ac357/users/0a19931bb6ad4e59ab61c7e36892d2e9/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:18.974680 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdedb70-91c6-4c0f-aef1-ab1f8355ebed admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1f27f395004d3fb0d34f05c39ac357, user_id=0a19931bb6ad4e59ab61c7e36892d2e9, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:18.977296 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdedb70-91c6-4c0f-aef1-ab1f8355ebed admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:18.987189 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1439/2868] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:57:18 2022] PUT /identity/v3/projects/7b1f27f395004d3fb0d34f05c39ac357/users/0a19931bb6ad4e59ab61c7e36892d2e9/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:18.997083 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89daaa41-9c70-4c56-90b2-da8308748b8b None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:18.997379 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89daaa41-9c70-4c56-90b2-da8308748b8b None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:18.997663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89daaa41-9c70-4c56-90b2-da8308748b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:19.023738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-89daaa41-9c70-4c56-90b2-da8308748b8b None None] MFA Rules not processed for user `0a19931bb6ad4e59ab61c7e36892d2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:19.024939 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-89daaa41-9c70-4c56-90b2-da8308748b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:19.098732 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1430/2869] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:18 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:57:19.110279 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdedb70-91c6-4c0f-aef1-ab1f8355ebed admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:19.111989 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-2bdedb70-91c6-4c0f-aef1-ab1f8355ebed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:19.167718 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdedb70-91c6-4c0f-aef1-ab1f8355ebed admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:19.171308 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e789e-e83e-4fb3-bc75-d211e929032f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:19.171946 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e789e-e83e-4fb3-bc75-d211e929032f service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:19.172231 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e789e-e83e-4fb3-bc75-d211e929032f service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:19.172505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e789e-e83e-4fb3-bc75-d211e929032f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:19.182163 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e789e-e83e-4fb3-bc75-d211e929032f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:19.183614 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e789e-e83e-4fb3-bc75-d211e929032f service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:19.193944 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1440/2870] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:57:19 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:57:20.038159 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89daaa41-9c70-4c56-90b2-da8308748b8b None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:20.098572 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89daaa41-9c70-4c56-90b2-da8308748b8b None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:20.103077 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47afa111-862f-4f8e-8a04-e3531d5dd7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:20.103936 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afa111-862f-4f8e-8a04-e3531d5dd7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:20.104217 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afa111-862f-4f8e-8a04-e3531d5dd7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:20.104492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afa111-862f-4f8e-8a04-e3531d5dd7c8 admin admin] PATH_INFO: `/v3/users/38200536129a4c27b6f8734590744763` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:20.105676 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47afa111-862f-4f8e-8a04-e3531d5dd7c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38200536129a4c27b6f8734590744763)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:20.107618 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47afa111-862f-4f8e-8a04-e3531d5dd7c8 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:20.296960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-47afa111-862f-4f8e-8a04-e3531d5dd7c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38200536129a4c27b6f8734590744763'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:20.297565 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-47afa111-862f-4f8e-8a04-e3531d5dd7c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38200536129a4c27b6f8734590744763'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:20.309261 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-47afa111-862f-4f8e-8a04-e3531d5dd7c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38200536129a4c27b6f8734590744763'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:20.309882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-47afa111-862f-4f8e-8a04-e3531d5dd7c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38200536129a4c27b6f8734590744763'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:20.316576 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-47afa111-862f-4f8e-8a04-e3531d5dd7c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38200536129a4c27b6f8734590744763'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:20.385896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1431/2871] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:57:20 2022] DELETE /identity/v3/users/38200536129a4c27b6f8734590744763 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:20.390416 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6e8c0-c9a4-4af6-9c2b-70992fbb06c3 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:20.390897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6e8c0-c9a4-4af6-9c2b-70992fbb06c3 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:20.391013 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6e8c0-c9a4-4af6-9c2b-70992fbb06c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:20.409632 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-73e6e8c0-c9a4-4af6-9c2b-70992fbb06c3 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:20.410866 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-73e6e8c0-c9a4-4af6-9c2b-70992fbb06c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:20.468009 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1441/2872] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:57:20.478242 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47afa111-862f-4f8e-8a04-e3531d5dd7c8 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:20.479626 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-47afa111-862f-4f8e-8a04-e3531d5dd7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:20.535476 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47afa111-862f-4f8e-8a04-e3531d5dd7c8 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:20.539163 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205d038-827b-45eb-8547-8434a00a9193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:20.539798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205d038-827b-45eb-8547-8434a00a9193 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:20.540079 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205d038-827b-45eb-8547-8434a00a9193 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:20.540353 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205d038-827b-45eb-8547-8434a00a9193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:20.549294 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205d038-827b-45eb-8547-8434a00a9193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:20.550704 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205d038-827b-45eb-8547-8434a00a9193 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:20.559782 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1432/2873] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:57:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:57:20.919113 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6e8c0-c9a4-4af6-9c2b-70992fbb06c3 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:20.920556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-73e6e8c0-c9a4-4af6-9c2b-70992fbb06c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:20.934386 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6e8c0-c9a4-4af6-9c2b-70992fbb06c3 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:20.939495 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44891ee7-9682-48a6-9308-d01146bfb03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:20.940346 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44891ee7-9682-48a6-9308-d01146bfb03d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:20.941165 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44891ee7-9682-48a6-9308-d01146bfb03d admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:20.941452 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44891ee7-9682-48a6-9308-d01146bfb03d admin admin] PATH_INFO: `/v3/projects/d4be45b752664f3c86a051d61e2fb947` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:20.942834 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44891ee7-9682-48a6-9308-d01146bfb03d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4be45b752664f3c86a051d61e2fb947)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:20.944723 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44891ee7-9682-48a6-9308-d01146bfb03d admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:21.048284 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-44891ee7-9682-48a6-9308-d01146bfb03d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4be45b752664f3c86a051d61e2fb947', 'request_id': 'req-44891ee7-9682-48a6-9308-d01146bfb03d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:21.123851 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-44891ee7-9682-48a6-9308-d01146bfb03d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4be45b752664f3c86a051d61e2fb947', 'request_id': 'req-44891ee7-9682-48a6-9308-d01146bfb03d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:21.130159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-44891ee7-9682-48a6-9308-d01146bfb03d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4be45b752664f3c86a051d61e2fb947', 'request_id': 'req-44891ee7-9682-48a6-9308-d01146bfb03d'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:21.140923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-44891ee7-9682-48a6-9308-d01146bfb03d admin admin] The token cache is being invalidate because project d4be45b752664f3c86a051d61e2fb947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:57:21.141209 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-44891ee7-9682-48a6-9308-d01146bfb03d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:21.141847 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-44891ee7-9682-48a6-9308-d01146bfb03d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:21.143420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1442/2874] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:57:20 2022] DELETE /identity/v3/projects/d4be45b752664f3c86a051d61e2fb947 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:21.147896 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205d038-827b-45eb-8547-8434a00a9193 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:21.157551 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205d038-827b-45eb-8547-8434a00a9193 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:21.211494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71801ded-5087-4fea-8bae-f4282cc77e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:21.212386 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71801ded-5087-4fea-8bae-f4282cc77e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:21.212669 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71801ded-5087-4fea-8bae-f4282cc77e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:21.212941 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71801ded-5087-4fea-8bae-f4282cc77e66 admin admin] PATH_INFO: `/v3/users/0a19931bb6ad4e59ab61c7e36892d2e9` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:21.214154 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71801ded-5087-4fea-8bae-f4282cc77e66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a19931bb6ad4e59ab61c7e36892d2e9)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:21.216057 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71801ded-5087-4fea-8bae-f4282cc77e66 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:21.387882 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-71801ded-5087-4fea-8bae-f4282cc77e66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a19931bb6ad4e59ab61c7e36892d2e9'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:21.388459 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-71801ded-5087-4fea-8bae-f4282cc77e66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a19931bb6ad4e59ab61c7e36892d2e9'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:21.400035 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-71801ded-5087-4fea-8bae-f4282cc77e66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a19931bb6ad4e59ab61c7e36892d2e9'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:21.400665 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-71801ded-5087-4fea-8bae-f4282cc77e66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a19931bb6ad4e59ab61c7e36892d2e9'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:21.407770 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-71801ded-5087-4fea-8bae-f4282cc77e66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a19931bb6ad4e59ab61c7e36892d2e9'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:21.477565 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1433/2875] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:57:21 2022] DELETE /identity/v3/users/0a19931bb6ad4e59ab61c7e36892d2e9 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:21.834157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44891ee7-9682-48a6-9308-d01146bfb03d admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:21.835603 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-44891ee7-9682-48a6-9308-d01146bfb03d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:21.849275 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44891ee7-9682-48a6-9308-d01146bfb03d admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:21.908022 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6ecf3-563b-4563-bfca-88dd3678bda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:21.908778 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6ecf3-563b-4563-bfca-88dd3678bda4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:21.909059 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6ecf3-563b-4563-bfca-88dd3678bda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:21.909330 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6ecf3-563b-4563-bfca-88dd3678bda4 admin admin] PATH_INFO: `/v3/projects/7b1f27f395004d3fb0d34f05c39ac357` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:21.910556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6ecf3-563b-4563-bfca-88dd3678bda4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b1f27f395004d3fb0d34f05c39ac357)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:21.912449 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6ecf3-563b-4563-bfca-88dd3678bda4 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:22.002844 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ddd6ecf3-563b-4563-bfca-88dd3678bda4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b1f27f395004d3fb0d34f05c39ac357', 'request_id': 'req-ddd6ecf3-563b-4563-bfca-88dd3678bda4'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:22.066378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ddd6ecf3-563b-4563-bfca-88dd3678bda4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b1f27f395004d3fb0d34f05c39ac357', 'request_id': 'req-ddd6ecf3-563b-4563-bfca-88dd3678bda4'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:22.071810 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ddd6ecf3-563b-4563-bfca-88dd3678bda4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b1f27f395004d3fb0d34f05c39ac357', 'request_id': 'req-ddd6ecf3-563b-4563-bfca-88dd3678bda4'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:22.083073 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ddd6ecf3-563b-4563-bfca-88dd3678bda4 admin admin] The token cache is being invalidate because project 7b1f27f395004d3fb0d34f05c39ac357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:57:22.083362 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ddd6ecf3-563b-4563-bfca-88dd3678bda4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:22.083986 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-ddd6ecf3-563b-4563-bfca-88dd3678bda4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:22.085519 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1443/2876] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:57:21 2022] DELETE /identity/v3/projects/7b1f27f395004d3fb0d34f05c39ac357 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:22.092320 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d19276-7ef2-43a6-885f-f95c960dd6c2 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:22.092614 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d19276-7ef2-43a6-885f-f95c960dd6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:22.092897 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d19276-7ef2-43a6-885f-f95c960dd6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:22.110919 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-a0d19276-7ef2-43a6-885f-f95c960dd6c2 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:22.112123 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-a0d19276-7ef2-43a6-885f-f95c960dd6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:22.169457 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1434/2877] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:57:22.178575 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b12ce1-f048-43bc-871e-98ae35b38ce8 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:22.178861 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b12ce1-f048-43bc-871e-98ae35b38ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:22.179135 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b12ce1-f048-43bc-871e-98ae35b38ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:22.198910 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-16b12ce1-f048-43bc-871e-98ae35b38ce8 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:22.200135 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-16b12ce1-f048-43bc-871e-98ae35b38ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:22.208543 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1444/2878] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:57:22.212870 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d19276-7ef2-43a6-885f-f95c960dd6c2 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:22.222515 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d19276-7ef2-43a6-885f-f95c960dd6c2 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:22.227096 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de74877-0af7-48fe-a969-e048353a8416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:22.227746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de74877-0af7-48fe-a969-e048353a8416 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:22.228030 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de74877-0af7-48fe-a969-e048353a8416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:22.228305 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de74877-0af7-48fe-a969-e048353a8416 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:22.229035 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de74877-0af7-48fe-a969-e048353a8416 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:57:22.229342 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de74877-0af7-48fe-a969-e048353a8416 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:22.230976 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de74877-0af7-48fe-a969-e048353a8416 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:22.242251 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1435/2879] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:57:22.246452 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b12ce1-f048-43bc-871e-98ae35b38ce8 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:22.256048 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b12ce1-f048-43bc-871e-98ae35b38ce8 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:22.260596 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73837ff-3606-4d35-959e-d5f50f8ea92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:22.261229 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73837ff-3606-4d35-959e-d5f50f8ea92e admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:22.261510 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73837ff-3606-4d35-959e-d5f50f8ea92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:22.261802 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73837ff-3606-4d35-959e-d5f50f8ea92e admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:22.262535 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73837ff-3606-4d35-959e-d5f50f8ea92e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:22.264407 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73837ff-3606-4d35-959e-d5f50f8ea92e admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:22.347460 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1445/2880] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:57:22 2022] POST /identity/v3/projects => generated 409 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:57:22.351535 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de74877-0af7-48fe-a969-e048353a8416 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:22.361303 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de74877-0af7-48fe-a969-e048353a8416 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:22.416441 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38947e-7938-4041-a02b-0714e230bcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:22.417109 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38947e-7938-4041-a02b-0714e230bcbf admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:22.417398 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38947e-7938-4041-a02b-0714e230bcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:22.417679 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38947e-7938-4041-a02b-0714e230bcbf admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:22.418401 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38947e-7938-4041-a02b-0714e230bcbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:22.420380 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38947e-7938-4041-a02b-0714e230bcbf admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:22.495659 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1436/2881] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:57:22 2022] POST /identity/v3/users => generated 305 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:57:22.499914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73837ff-3606-4d35-959e-d5f50f8ea92e admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:22.511157 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73837ff-3606-4d35-959e-d5f50f8ea92e admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:22.516298 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada166e-7493-4f2d-86af-09e5a4b9b6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:22.520491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada166e-7493-4f2d-86af-09e5a4b9b6ba admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:22.520491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada166e-7493-4f2d-86af-09e5a4b9b6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:22.520491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada166e-7493-4f2d-86af-09e5a4b9b6ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:22.520491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada166e-7493-4f2d-86af-09e5a4b9b6ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:22.520873 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada166e-7493-4f2d-86af-09e5a4b9b6ba admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:22.531800 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1446/2882] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:57:22.535807 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38947e-7938-4041-a02b-0714e230bcbf admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:22.545674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38947e-7938-4041-a02b-0714e230bcbf admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:22.550754 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9a151-2856-4bc9-a56e-df7e3e48e485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:22.551396 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9a151-2856-4bc9-a56e-df7e3e48e485 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:22.551680 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9a151-2856-4bc9-a56e-df7e3e48e485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:22.551955 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9a151-2856-4bc9-a56e-df7e3e48e485 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:22.552678 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9a151-2856-4bc9-a56e-df7e3e48e485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:22.554305 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9a151-2856-4bc9-a56e-df7e3e48e485 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:22.564786 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1437/2883] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:57:22.568629 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada166e-7493-4f2d-86af-09e5a4b9b6ba admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:22.578420 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada166e-7493-4f2d-86af-09e5a4b9b6ba admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:22.583000 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8db981-1078-447d-8a3e-46009ba2b208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:22.583860 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8db981-1078-447d-8a3e-46009ba2b208 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:22.584142 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8db981-1078-447d-8a3e-46009ba2b208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:22.584417 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8db981-1078-447d-8a3e-46009ba2b208 admin admin] PATH_INFO: `/v3/projects/f2cd5c59176947dc93e8e186e4e889a2/users/b3fd38d9a616446fb142782724d4759b/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:22.598342 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8db981-1078-447d-8a3e-46009ba2b208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2cd5c59176947dc93e8e186e4e889a2, user_id=b3fd38d9a616446fb142782724d4759b, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:22.600967 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8db981-1078-447d-8a3e-46009ba2b208 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:22.683443 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1447/2884] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:57:22 2022] PUT /identity/v3/projects/f2cd5c59176947dc93e8e186e4e889a2/users/b3fd38d9a616446fb142782724d4759b/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:57:22.694326 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9feca-7102-47f6-8126-922e433c9236 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:22.694623 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9feca-7102-47f6-8126-922e433c9236 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:22.694909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9feca-7102-47f6-8126-922e433c9236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:22.720554 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-acd9feca-7102-47f6-8126-922e433c9236 None None] MFA Rules not processed for user `b3fd38d9a616446fb142782724d4759b`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:22.721761 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-acd9feca-7102-47f6-8126-922e433c9236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:22.783837 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1438/2885] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:22 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:57:22.788416 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8db981-1078-447d-8a3e-46009ba2b208 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:22.849485 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8db981-1078-447d-8a3e-46009ba2b208 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:22.854174 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a842dc3-337a-43ff-b12c-57cedac0b474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:22.854826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a842dc3-337a-43ff-b12c-57cedac0b474 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:22.855108 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a842dc3-337a-43ff-b12c-57cedac0b474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:22.855393 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a842dc3-337a-43ff-b12c-57cedac0b474 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:22.856097 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a842dc3-337a-43ff-b12c-57cedac0b474 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:22.857966 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a842dc3-337a-43ff-b12c-57cedac0b474 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:22.964792 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1448/2886] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:57:22 2022] POST /identity/v3/projects => generated 409 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:57:22.968880 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9feca-7102-47f6-8126-922e433c9236 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:22.980144 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9feca-7102-47f6-8126-922e433c9236 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:23.034454 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1fd9a-3705-4853-8c04-7824f4bb053d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:23.035125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1fd9a-3705-4853-8c04-7824f4bb053d admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:23.035411 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1fd9a-3705-4853-8c04-7824f4bb053d admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:23.035691 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1fd9a-3705-4853-8c04-7824f4bb053d admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:23.036394 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1fd9a-3705-4853-8c04-7824f4bb053d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:23.038323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1fd9a-3705-4853-8c04-7824f4bb053d admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:23.124351 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1439/2887] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:57:22 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:57:23.129380 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a842dc3-337a-43ff-b12c-57cedac0b474 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:23.139813 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a842dc3-337a-43ff-b12c-57cedac0b474 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:23.144626 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802562c5-5831-4d70-b10c-dcd4ee9b06f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:23.145559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802562c5-5831-4d70-b10c-dcd4ee9b06f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:23.145839 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802562c5-5831-4d70-b10c-dcd4ee9b06f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:23.146149 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802562c5-5831-4d70-b10c-dcd4ee9b06f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:23.146897 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802562c5-5831-4d70-b10c-dcd4ee9b06f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:23.148514 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802562c5-5831-4d70-b10c-dcd4ee9b06f0 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:23.159959 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1449/2888] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 20:57:23.164192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1fd9a-3705-4853-8c04-7824f4bb053d admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:23.174943 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1fd9a-3705-4853-8c04-7824f4bb053d admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:23.179586 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13ac96-c28b-4251-a36e-4f42abf28afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:23.180460 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13ac96-c28b-4251-a36e-4f42abf28afa admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:23.180746 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13ac96-c28b-4251-a36e-4f42abf28afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:23.181025 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13ac96-c28b-4251-a36e-4f42abf28afa admin admin] PATH_INFO: `/v3/projects/5b0daf8e208140bdb34ea86a42105eb3/users/21533155262546dca69f603b039add1e/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:23.195394 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13ac96-c28b-4251-a36e-4f42abf28afa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b0daf8e208140bdb34ea86a42105eb3, user_id=21533155262546dca69f603b039add1e, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:23.198073 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13ac96-c28b-4251-a36e-4f42abf28afa admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:23.260845 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1440/2889] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:57:23 2022] PUT /identity/v3/projects/5b0daf8e208140bdb34ea86a42105eb3/users/21533155262546dca69f603b039add1e/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:23.265018 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802562c5-5831-4d70-b10c-dcd4ee9b06f0 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:23.276058 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802562c5-5831-4d70-b10c-dcd4ee9b06f0 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:23.330356 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71127a3c-4eeb-41c9-8dd2-8e417fb222b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:23.331021 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71127a3c-4eeb-41c9-8dd2-8e417fb222b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:23.331309 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71127a3c-4eeb-41c9-8dd2-8e417fb222b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:23.331591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71127a3c-4eeb-41c9-8dd2-8e417fb222b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:23.332326 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71127a3c-4eeb-41c9-8dd2-8e417fb222b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:23.333920 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71127a3c-4eeb-41c9-8dd2-8e417fb222b8 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:23.343366 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1450/2890] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:57:23.348093 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13ac96-c28b-4251-a36e-4f42abf28afa admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:23.358173 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13ac96-c28b-4251-a36e-4f42abf28afa admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:23.362963 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e1b8d-ce5f-49d8-8098-40add5cbc617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:23.363868 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e1b8d-ce5f-49d8-8098-40add5cbc617 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:23.373273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e1b8d-ce5f-49d8-8098-40add5cbc617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:23.373273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e1b8d-ce5f-49d8-8098-40add5cbc617 admin admin] PATH_INFO: `/v3/projects/5b0daf8e208140bdb34ea86a42105eb3/users/21533155262546dca69f603b039add1e/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:23.373273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e1b8d-ce5f-49d8-8098-40add5cbc617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b0daf8e208140bdb34ea86a42105eb3, user_id=21533155262546dca69f603b039add1e, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:23.373273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e1b8d-ce5f-49d8-8098-40add5cbc617 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:23.379280 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1441/2891] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:57:23 2022] PUT /identity/v3/projects/5b0daf8e208140bdb34ea86a42105eb3/users/21533155262546dca69f603b039add1e/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:23.389148 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401265bd-32b7-49e7-ba30-5b5ef8cb2a28 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:23.389559 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401265bd-32b7-49e7-ba30-5b5ef8cb2a28 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:23.389864 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401265bd-32b7-49e7-ba30-5b5ef8cb2a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:23.417043 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-401265bd-32b7-49e7-ba30-5b5ef8cb2a28 None None] MFA Rules not processed for user `21533155262546dca69f603b039add1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:23.418297 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-401265bd-32b7-49e7-ba30-5b5ef8cb2a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:23.484508 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1451/2892] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:23 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 20:57:23.494991 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e1b8d-ce5f-49d8-8098-40add5cbc617 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:23.496645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-7a1e1b8d-ce5f-49d8-8098-40add5cbc617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:23.552216 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e1b8d-ce5f-49d8-8098-40add5cbc617 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:23.555748 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444abc3-101e-487a-a968-034d2ab167f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:23.556367 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444abc3-101e-487a-a968-034d2ab167f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:23.556645 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444abc3-101e-487a-a968-034d2ab167f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:23.557003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444abc3-101e-487a-a968-034d2ab167f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:23.565740 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5444abc3-101e-487a-a968-034d2ab167f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:23.567172 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5444abc3-101e-487a-a968-034d2ab167f6 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:23.576265 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1442/2893] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:57:23 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:57:23.868505 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401265bd-32b7-49e7-ba30-5b5ef8cb2a28 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:23.878893 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401265bd-32b7-49e7-ba30-5b5ef8cb2a28 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:23.882499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43655a34-5b5e-4cfe-b994-d39d1f4fefd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:23.883132 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43655a34-5b5e-4cfe-b994-d39d1f4fefd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:23.883472 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43655a34-5b5e-4cfe-b994-d39d1f4fefd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:23.883753 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43655a34-5b5e-4cfe-b994-d39d1f4fefd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:23.940510 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43655a34-5b5e-4cfe-b994-d39d1f4fefd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:23.941933 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43655a34-5b5e-4cfe-b994-d39d1f4fefd9 service cinder] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:23.950886 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1452/2894] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:57:23 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:57:33.581099 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444abc3-101e-487a-a968-034d2ab167f6 service cinder] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:33.643003 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444abc3-101e-487a-a968-034d2ab167f6 service cinder] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:33.647905 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b6aa6-ff6b-444f-95cf-406f02b2f1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:33.648762 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b6aa6-ff6b-444f-95cf-406f02b2f1f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:33.649044 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b6aa6-ff6b-444f-95cf-406f02b2f1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:33.649323 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b6aa6-ff6b-444f-95cf-406f02b2f1f9 admin admin] PATH_INFO: `/v3/users/b3fd38d9a616446fb142782724d4759b` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:33.650553 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b6aa6-ff6b-444f-95cf-406f02b2f1f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3fd38d9a616446fb142782724d4759b)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:33.652474 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b6aa6-ff6b-444f-95cf-406f02b2f1f9 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:33.810978 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-fc1b6aa6-ff6b-444f-95cf-406f02b2f1f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3fd38d9a616446fb142782724d4759b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:33.811626 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-fc1b6aa6-ff6b-444f-95cf-406f02b2f1f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3fd38d9a616446fb142782724d4759b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:33.823257 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-fc1b6aa6-ff6b-444f-95cf-406f02b2f1f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3fd38d9a616446fb142782724d4759b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:33.823976 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-fc1b6aa6-ff6b-444f-95cf-406f02b2f1f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3fd38d9a616446fb142782724d4759b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:33.831385 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-fc1b6aa6-ff6b-444f-95cf-406f02b2f1f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3fd38d9a616446fb142782724d4759b'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:33.900041 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1443/2895] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:57:33 2022] DELETE /identity/v3/users/b3fd38d9a616446fb142782724d4759b => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:33.904591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0e250-78c5-4b83-b251-80fd823b2844 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:33.905058 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0e250-78c5-4b83-b251-80fd823b2844 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:33.905166 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0e250-78c5-4b83-b251-80fd823b2844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:33.922766 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-2ed0e250-78c5-4b83-b251-80fd823b2844 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:33.923954 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-2ed0e250-78c5-4b83-b251-80fd823b2844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:33.980158 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1453/2896] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:57:33.989784 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b6aa6-ff6b-444f-95cf-406f02b2f1f9 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:33.991228 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-fc1b6aa6-ff6b-444f-95cf-406f02b2f1f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:34.046629 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b6aa6-ff6b-444f-95cf-406f02b2f1f9 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:34.050394 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a5464-1fcc-43bb-9023-6f7c88544e39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:34.051034 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a5464-1fcc-43bb-9023-6f7c88544e39 service neutron] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:34.051314 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a5464-1fcc-43bb-9023-6f7c88544e39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:34.051588 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a5464-1fcc-43bb-9023-6f7c88544e39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:34.060459 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a5464-1fcc-43bb-9023-6f7c88544e39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:34.061935 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a5464-1fcc-43bb-9023-6f7c88544e39 service neutron] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:34.071674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1444/2897] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:57:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:57:34.462478 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0e250-78c5-4b83-b251-80fd823b2844 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:34.463959 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-2ed0e250-78c5-4b83-b251-80fd823b2844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:34.475876 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0e250-78c5-4b83-b251-80fd823b2844 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:34.480792 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cce91-4fe9-4934-a229-237a9e2d7391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:34.484117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cce91-4fe9-4934-a229-237a9e2d7391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:34.484117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cce91-4fe9-4934-a229-237a9e2d7391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:34.484117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cce91-4fe9-4934-a229-237a9e2d7391 admin admin] PATH_INFO: `/v3/projects/f2cd5c59176947dc93e8e186e4e889a2` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:34.484117 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cce91-4fe9-4934-a229-237a9e2d7391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2cd5c59176947dc93e8e186e4e889a2)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:34.485831 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cce91-4fe9-4934-a229-237a9e2d7391 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:34.568125 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-4b5cce91-4fe9-4934-a229-237a9e2d7391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2cd5c59176947dc93e8e186e4e889a2', 'request_id': 'req-4b5cce91-4fe9-4934-a229-237a9e2d7391'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:34.614465 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-4b5cce91-4fe9-4934-a229-237a9e2d7391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2cd5c59176947dc93e8e186e4e889a2', 'request_id': 'req-4b5cce91-4fe9-4934-a229-237a9e2d7391'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:34.619863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-4b5cce91-4fe9-4934-a229-237a9e2d7391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2cd5c59176947dc93e8e186e4e889a2', 'request_id': 'req-4b5cce91-4fe9-4934-a229-237a9e2d7391'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:34.631251 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-4b5cce91-4fe9-4934-a229-237a9e2d7391 admin admin] The token cache is being invalidate because project f2cd5c59176947dc93e8e186e4e889a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:57:34.633122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-4b5cce91-4fe9-4934-a229-237a9e2d7391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:34.633122 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-4b5cce91-4fe9-4934-a229-237a9e2d7391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:34.633685 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1454/2898] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:57:34 2022] DELETE /identity/v3/projects/f2cd5c59176947dc93e8e186e4e889a2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:34.637793 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a5464-1fcc-43bb-9023-6f7c88544e39 service neutron] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:34.647515 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a5464-1fcc-43bb-9023-6f7c88544e39 service neutron] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:34.702810 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a4f94-5cf2-4638-821d-49c89080849f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:34.703673 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a4f94-5cf2-4638-821d-49c89080849f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:34.703958 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a4f94-5cf2-4638-821d-49c89080849f admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:34.704234 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a4f94-5cf2-4638-821d-49c89080849f admin admin] PATH_INFO: `/v3/users/21533155262546dca69f603b039add1e` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:34.705427 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a4f94-5cf2-4638-821d-49c89080849f admin admin] RBAC: Authorizing `identity:delete_user(user_id=21533155262546dca69f603b039add1e)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:34.707359 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a4f94-5cf2-4638-821d-49c89080849f admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:34.850690 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-163a4f94-5cf2-4638-821d-49c89080849f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21533155262546dca69f603b039add1e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:34.851319 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-163a4f94-5cf2-4638-821d-49c89080849f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21533155262546dca69f603b039add1e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:34.863237 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-163a4f94-5cf2-4638-821d-49c89080849f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21533155262546dca69f603b039add1e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:34.863849 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-163a4f94-5cf2-4638-821d-49c89080849f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21533155262546dca69f603b039add1e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:34.870449 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-163a4f94-5cf2-4638-821d-49c89080849f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21533155262546dca69f603b039add1e'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:34.939890 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1445/2899] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:57:34 2022] DELETE /identity/v3/users/21533155262546dca69f603b039add1e => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:57:35.347733 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cce91-4fe9-4934-a229-237a9e2d7391 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:35.349172 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-4b5cce91-4fe9-4934-a229-237a9e2d7391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:35.364250 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cce91-4fe9-4934-a229-237a9e2d7391 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:35.421544 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c442a-e707-4d7b-bc47-030becc795ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:35.422364 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c442a-e707-4d7b-bc47-030becc795ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:35.422647 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c442a-e707-4d7b-bc47-030becc795ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:35.422947 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c442a-e707-4d7b-bc47-030becc795ba admin admin] PATH_INFO: `/v3/projects/5b0daf8e208140bdb34ea86a42105eb3` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:35.424139 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c442a-e707-4d7b-bc47-030becc795ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b0daf8e208140bdb34ea86a42105eb3)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:35.426098 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c442a-e707-4d7b-bc47-030becc795ba admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:35.522507 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-bf9c442a-e707-4d7b-bc47-030becc795ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b0daf8e208140bdb34ea86a42105eb3', 'request_id': 'req-bf9c442a-e707-4d7b-bc47-030becc795ba'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:35.591061 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-bf9c442a-e707-4d7b-bc47-030becc795ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b0daf8e208140bdb34ea86a42105eb3', 'request_id': 'req-bf9c442a-e707-4d7b-bc47-030becc795ba'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:35.596378 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-bf9c442a-e707-4d7b-bc47-030becc795ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b0daf8e208140bdb34ea86a42105eb3', 'request_id': 'req-bf9c442a-e707-4d7b-bc47-030becc795ba'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:35.606963 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-bf9c442a-e707-4d7b-bc47-030becc795ba admin admin] The token cache is being invalidate because project 5b0daf8e208140bdb34ea86a42105eb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80515) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:57:35.607248 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-bf9c442a-e707-4d7b-bc47-030becc795ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:35.607863 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-bf9c442a-e707-4d7b-bc47-030becc795ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:35.609387 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1455/2900] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:57:35 2022] DELETE /identity/v3/projects/5b0daf8e208140bdb34ea86a42105eb3 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:35.614666 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9664d68-6e3f-4564-a40a-f3e1c49153da None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:35.614960 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9664d68-6e3f-4564-a40a-f3e1c49153da None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:35.615243 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9664d68-6e3f-4564-a40a-f3e1c49153da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:35.632909 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-f9664d68-6e3f-4564-a40a-f3e1c49153da None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:35.634172 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-f9664d68-6e3f-4564-a40a-f3e1c49153da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:35.691912 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1446/2901] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:57:35.700977 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320251f-344b-4fd5-a0b1-641cbd35ebf3 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:35.701256 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320251f-344b-4fd5-a0b1-641cbd35ebf3 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:35.701536 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320251f-344b-4fd5-a0b1-641cbd35ebf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:35.720003 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-1320251f-344b-4fd5-a0b1-641cbd35ebf3 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:35.721206 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-1320251f-344b-4fd5-a0b1-641cbd35ebf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:35.729129 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1456/2902] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:57:35.733260 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9664d68-6e3f-4564-a40a-f3e1c49153da None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:35.742894 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9664d68-6e3f-4564-a40a-f3e1c49153da None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:35.747494 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278eecf-67fa-4abc-9997-95b8ed98e003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:35.748281 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278eecf-67fa-4abc-9997-95b8ed98e003 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:35.748565 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278eecf-67fa-4abc-9997-95b8ed98e003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:35.748839 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278eecf-67fa-4abc-9997-95b8ed98e003 admin admin] PATH_INFO: `/v3/domains` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:35.749562 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3278eecf-67fa-4abc-9997-95b8ed98e003 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80516) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 20:57:35.749867 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3278eecf-67fa-4abc-9997-95b8ed98e003 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:35.751995 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3278eecf-67fa-4abc-9997-95b8ed98e003 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:35.763337 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1447/2903] 10.222.0.65 () {58 vars in 1152 bytes} [Tue May 10 20:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 20:57:35.767860 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320251f-344b-4fd5-a0b1-641cbd35ebf3 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:35.777068 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320251f-344b-4fd5-a0b1-641cbd35ebf3 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:35.781872 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff957bd-5518-46c4-bca5-121f17060b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:35.782558 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff957bd-5518-46c4-bca5-121f17060b09 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:35.782839 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff957bd-5518-46c4-bca5-121f17060b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:35.783115 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff957bd-5518-46c4-bca5-121f17060b09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:35.783811 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff957bd-5518-46c4-bca5-121f17060b09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:35.785708 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff957bd-5518-46c4-bca5-121f17060b09 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:35.896644 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1457/2904] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:57:35 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:57:35.900749 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278eecf-67fa-4abc-9997-95b8ed98e003 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:35.910605 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278eecf-67fa-4abc-9997-95b8ed98e003 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:35.965798 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f996cb6-d85d-4f5d-88dc-38f8462d065f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:35.966498 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f996cb6-d85d-4f5d-88dc-38f8462d065f admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:35.966779 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f996cb6-d85d-4f5d-88dc-38f8462d065f admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:35.967049 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f996cb6-d85d-4f5d-88dc-38f8462d065f admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:35.967738 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f996cb6-d85d-4f5d-88dc-38f8462d065f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:35.969735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f996cb6-d85d-4f5d-88dc-38f8462d065f admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:36.043089 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1448/2905] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:57:35 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:57:36.047154 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff957bd-5518-46c4-bca5-121f17060b09 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:36.057856 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff957bd-5518-46c4-bca5-121f17060b09 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:36.062551 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938cd065-4128-402c-8c95-15e050155445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:36.063203 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cd065-4128-402c-8c95-15e050155445 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:36.064033 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cd065-4128-402c-8c95-15e050155445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:36.064313 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cd065-4128-402c-8c95-15e050155445 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:36.065050 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938cd065-4128-402c-8c95-15e050155445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:36.066666 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938cd065-4128-402c-8c95-15e050155445 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:36.076671 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1458/2906] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:57:36.080478 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f996cb6-d85d-4f5d-88dc-38f8462d065f admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:36.090412 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f996cb6-d85d-4f5d-88dc-38f8462d065f admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:36.095175 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877afb60-92a7-4b85-96ea-9df9f1abcd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:36.095820 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877afb60-92a7-4b85-96ea-9df9f1abcd02 admin admin] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:36.096104 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877afb60-92a7-4b85-96ea-9df9f1abcd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:36.096381 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877afb60-92a7-4b85-96ea-9df9f1abcd02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:36.097106 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877afb60-92a7-4b85-96ea-9df9f1abcd02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:36.098735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877afb60-92a7-4b85-96ea-9df9f1abcd02 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:36.108744 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1449/2907] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:57:36.113162 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938cd065-4128-402c-8c95-15e050155445 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:36.122841 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938cd065-4128-402c-8c95-15e050155445 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:36.128143 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64a3b7-4eac-462d-982d-0c35b6e26bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:36.129008 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64a3b7-4eac-462d-982d-0c35b6e26bd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:36.129291 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64a3b7-4eac-462d-982d-0c35b6e26bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:36.129571 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64a3b7-4eac-462d-982d-0c35b6e26bd0 admin admin] PATH_INFO: `/v3/projects/97074b588da542718abb3ea53aed39df/users/801fd42d82bb4a659b581eea082e1e15/roles/5f62ce25636443f8a5add67f208afe62` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:36.143460 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64a3b7-4eac-462d-982d-0c35b6e26bd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97074b588da542718abb3ea53aed39df, user_id=801fd42d82bb4a659b581eea082e1e15, role_id=5f62ce25636443f8a5add67f208afe62)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:36.146360 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64a3b7-4eac-462d-982d-0c35b6e26bd0 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:36.214409 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1459/2908] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:57:36 2022] PUT /identity/v3/projects/97074b588da542718abb3ea53aed39df/users/801fd42d82bb4a659b581eea082e1e15/roles/5f62ce25636443f8a5add67f208afe62 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:36.224475 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8627a4-6dae-41aa-bfcb-e719393b16f8 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:36.224827 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8627a4-6dae-41aa-bfcb-e719393b16f8 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:36.225125 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8627a4-6dae-41aa-bfcb-e719393b16f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:36.251901 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-0a8627a4-6dae-41aa-bfcb-e719393b16f8 None None] MFA Rules not processed for user `801fd42d82bb4a659b581eea082e1e15`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:36.253103 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-0a8627a4-6dae-41aa-bfcb-e719393b16f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:36.315189 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1450/2909] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:57:36.319556 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64a3b7-4eac-462d-982d-0c35b6e26bd0 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:36.377693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64a3b7-4eac-462d-982d-0c35b6e26bd0 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:36.382391 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d82d2-ca48-4502-96d6-3ee24a6950f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:36.383042 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d82d2-ca48-4502-96d6-3ee24a6950f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:36.383330 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d82d2-ca48-4502-96d6-3ee24a6950f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:36.383610 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d82d2-ca48-4502-96d6-3ee24a6950f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:36.384306 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d82d2-ca48-4502-96d6-3ee24a6950f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:36.386228 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d82d2-ca48-4502-96d6-3ee24a6950f2 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:36.467396 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1460/2910] 10.222.0.65 () {60 vars in 1152 bytes} [Tue May 10 20:57:36 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 20:57:36.471811 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8627a4-6dae-41aa-bfcb-e719393b16f8 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:36.482625 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8627a4-6dae-41aa-bfcb-e719393b16f8 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:36.536908 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc5fe4-2eed-49f3-8701-9ea004205bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:36.537566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc5fe4-2eed-49f3-8701-9ea004205bb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:36.537848 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc5fe4-2eed-49f3-8701-9ea004205bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:36.538149 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc5fe4-2eed-49f3-8701-9ea004205bb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:36.538850 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc5fe4-2eed-49f3-8701-9ea004205bb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:36.540740 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc5fe4-2eed-49f3-8701-9ea004205bb9 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:36.642980 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1451/2911] 10.222.0.65 () {60 vars in 1143 bytes} [Tue May 10 20:57:36 2022] POST /identity/v3/users => generated 300 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 20:57:36.646914 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d82d2-ca48-4502-96d6-3ee24a6950f2 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:36.657606 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d82d2-ca48-4502-96d6-3ee24a6950f2 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:36.662269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc64ae-330a-4937-883e-c701af9132b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:36.662919 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc64ae-330a-4937-883e-c701af9132b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:36.663205 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc64ae-330a-4937-883e-c701af9132b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:36.663484 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc64ae-330a-4937-883e-c701af9132b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:36.664218 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc64ae-330a-4937-883e-c701af9132b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:36.665803 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc64ae-330a-4937-883e-c701af9132b4 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:36.676640 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1461/2912] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:57:36.680619 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc5fe4-2eed-49f3-8701-9ea004205bb9 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:36.690481 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc5fe4-2eed-49f3-8701-9ea004205bb9 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:36.695082 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f93f3-8038-4f4c-984e-6c6652710ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:36.695933 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f93f3-8038-4f4c-984e-6c6652710ea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:36.696215 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f93f3-8038-4f4c-984e-6c6652710ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:36.696492 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f93f3-8038-4f4c-984e-6c6652710ea4 admin admin] PATH_INFO: `/v3/projects/37af467a93fd40df9b88069558af7e4a/users/38943e721ed54818b9844d8c4ffeea24/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:36.711192 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f93f3-8038-4f4c-984e-6c6652710ea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37af467a93fd40df9b88069558af7e4a, user_id=38943e721ed54818b9844d8c4ffeea24, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:36.713835 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f93f3-8038-4f4c-984e-6c6652710ea4 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:36.780221 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1452/2913] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:57:36 2022] PUT /identity/v3/projects/37af467a93fd40df9b88069558af7e4a/users/38943e721ed54818b9844d8c4ffeea24/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:57:36.784128 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc64ae-330a-4937-883e-c701af9132b4 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:36.794631 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc64ae-330a-4937-883e-c701af9132b4 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:36.848843 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac6c74-6ff5-4590-b251-64a62c6e77d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:36.849531 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac6c74-6ff5-4590-b251-64a62c6e77d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:36.849822 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac6c74-6ff5-4590-b251-64a62c6e77d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:36.850119 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac6c74-6ff5-4590-b251-64a62c6e77d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:36.850855 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac6c74-6ff5-4590-b251-64a62c6e77d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:36.852446 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac6c74-6ff5-4590-b251-64a62c6e77d4 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:36.862292 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1462/2914] 10.222.0.65 () {58 vars in 1121 bytes} [Tue May 10 20:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 20:57:36.866380 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f93f3-8038-4f4c-984e-6c6652710ea4 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:36.876455 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f93f3-8038-4f4c-984e-6c6652710ea4 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:36.881091 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c35d5f-7987-4f8c-821c-348e438e2015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:36.881962 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c35d5f-7987-4f8c-821c-348e438e2015 admin admin] REQUEST_METHOD: `PUT` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:36.882273 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c35d5f-7987-4f8c-821c-348e438e2015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:36.882551 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c35d5f-7987-4f8c-821c-348e438e2015 admin admin] PATH_INFO: `/v3/projects/37af467a93fd40df9b88069558af7e4a/users/38943e721ed54818b9844d8c4ffeea24/roles/5c87e804d0764309ba08ad6e06b23a87` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:36.884538 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c35d5f-7987-4f8c-821c-348e438e2015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37af467a93fd40df9b88069558af7e4a, user_id=38943e721ed54818b9844d8c4ffeea24, role_id=5c87e804d0764309ba08ad6e06b23a87)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:36.887193 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c35d5f-7987-4f8c-821c-348e438e2015 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:36.898040 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1453/2915] 10.222.0.65 () {60 vars in 1482 bytes} [Tue May 10 20:57:36 2022] PUT /identity/v3/projects/37af467a93fd40df9b88069558af7e4a/users/38943e721ed54818b9844d8c4ffeea24/roles/5c87e804d0764309ba08ad6e06b23a87 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 20:57:36.909602 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a3118-6136-4cb7-bfc5-6bc80e3020b2 None None] REQUEST_METHOD: `POST` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:36.909900 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a3118-6136-4cb7-bfc5-6bc80e3020b2 None None] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:36.910213 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a3118-6136-4cb7-bfc5-6bc80e3020b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:36.935927 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.auth.core [None req-ca2a3118-6136-4cb7-bfc5-6bc80e3020b2 None None] MFA Rules not processed for user `38943e721ed54818b9844d8c4ffeea24`. Rule list: `[]` (Enabled: `True`). {{(pid=80515) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:36.937159 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-ca2a3118-6136-4cb7-bfc5-6bc80e3020b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:37.001857 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1463/2916] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:57:37.012316 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c35d5f-7987-4f8c-821c-348e438e2015 admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:37.013968 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-54c35d5f-7987-4f8c-821c-348e438e2015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:37.071578 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c35d5f-7987-4f8c-821c-348e438e2015 admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:37.075162 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d88400-b205-4658-9fc5-8df9582008d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e366ec240e5464d8c5e92f807123c23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:37.075794 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d88400-b205-4658-9fc5-8df9582008d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:37.076078 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d88400-b205-4658-9fc5-8df9582008d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:37.076354 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d88400-b205-4658-9fc5-8df9582008d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:37.085663 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d88400-b205-4658-9fc5-8df9582008d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:37.087118 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d88400-b205-4658-9fc5-8df9582008d0 service cinder] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:37.096417 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1454/2917] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:57:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:57:38.045439 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a3118-6136-4cb7-bfc5-6bc80e3020b2 None None] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:38.104625 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a3118-6136-4cb7-bfc5-6bc80e3020b2 None None] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:38.109045 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f6ac6-80cb-4bc8-a894-0cec74e2fd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:38.109895 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f6ac6-80cb-4bc8-a894-0cec74e2fd8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:38.110215 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f6ac6-80cb-4bc8-a894-0cec74e2fd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:38.110491 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f6ac6-80cb-4bc8-a894-0cec74e2fd8b admin admin] PATH_INFO: `/v3/users/801fd42d82bb4a659b581eea082e1e15` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:38.111686 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f6ac6-80cb-4bc8-a894-0cec74e2fd8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=801fd42d82bb4a659b581eea082e1e15)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:38.113591 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f6ac6-80cb-4bc8-a894-0cec74e2fd8b admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:38.255923 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-610f6ac6-80cb-4bc8-a894-0cec74e2fd8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '801fd42d82bb4a659b581eea082e1e15'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:38.256576 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-610f6ac6-80cb-4bc8-a894-0cec74e2fd8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '801fd42d82bb4a659b581eea082e1e15'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:38.257177 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-610f6ac6-80cb-4bc8-a894-0cec74e2fd8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '801fd42d82bb4a659b581eea082e1e15'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:38.268637 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-610f6ac6-80cb-4bc8-a894-0cec74e2fd8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '801fd42d82bb4a659b581eea082e1e15'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:38.275474 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-610f6ac6-80cb-4bc8-a894-0cec74e2fd8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '801fd42d82bb4a659b581eea082e1e15'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:38.352181 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1464/2918] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:57:38 2022] DELETE /identity/v3/users/801fd42d82bb4a659b581eea082e1e15 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:38.357345 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd3473-5aad-48cc-a0b2-ee8392b8f388 None None] REQUEST_METHOD: `POST` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:38.357816 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd3473-5aad-48cc-a0b2-ee8392b8f388 None None] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:38.357939 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd3473-5aad-48cc-a0b2-ee8392b8f388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:38.375428 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.auth.core [None req-33cd3473-5aad-48cc-a0b2-ee8392b8f388 None None] MFA Rules not processed for user `729fa49ea5f943b2b09c35b636e34ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80516) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 20:57:38.377287 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-33cd3473-5aad-48cc-a0b2-ee8392b8f388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:38.434022 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1455/2919] 10.222.0.65 () {58 vars in 957 bytes} [Tue May 10 20:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 20:57:38.444237 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f6ac6-80cb-4bc8-a894-0cec74e2fd8b admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:38.445660 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-610f6ac6-80cb-4bc8-a894-0cec74e2fd8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:38.502240 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f6ac6-80cb-4bc8-a894-0cec74e2fd8b admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:38.505890 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e95f77-6723-4c76-9550-f3ba02059a14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6aabd1571d04bcfae842626fef5ed6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:38.506615 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e95f77-6723-4c76-9550-f3ba02059a14 service neutron] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:38.506849 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e95f77-6723-4c76-9550-f3ba02059a14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:38.507125 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e95f77-6723-4c76-9550-f3ba02059a14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:38.515996 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e95f77-6723-4c76-9550-f3ba02059a14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:38.517408 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e95f77-6723-4c76-9550-f3ba02059a14 service neutron] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:38.526909 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1465/2920] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 20:57:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 20:57:38.889668 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd3473-5aad-48cc-a0b2-ee8392b8f388 None None] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:38.891100 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-33cd3473-5aad-48cc-a0b2-ee8392b8f388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:38.908958 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd3473-5aad-48cc-a0b2-ee8392b8f388 None None] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:38.913699 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbf2bd-42ea-433d-951a-a1b722fc026a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:38.914879 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbf2bd-42ea-433d-951a-a1b722fc026a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:38.915155 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbf2bd-42ea-433d-951a-a1b722fc026a admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:38.915496 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbf2bd-42ea-433d-951a-a1b722fc026a admin admin] PATH_INFO: `/v3/projects/97074b588da542718abb3ea53aed39df` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:38.916674 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bbf2bd-42ea-433d-951a-a1b722fc026a admin admin] RBAC: Authorizing `identity:delete_project(project_id=97074b588da542718abb3ea53aed39df)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:38.918566 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bbf2bd-42ea-433d-951a-a1b722fc026a admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:39.013151 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-08bbf2bd-42ea-433d-951a-a1b722fc026a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97074b588da542718abb3ea53aed39df', 'request_id': 'req-08bbf2bd-42ea-433d-951a-a1b722fc026a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:39.071286 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-08bbf2bd-42ea-433d-951a-a1b722fc026a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97074b588da542718abb3ea53aed39df', 'request_id': 'req-08bbf2bd-42ea-433d-951a-a1b722fc026a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:39.076780 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-08bbf2bd-42ea-433d-951a-a1b722fc026a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97074b588da542718abb3ea53aed39df', 'request_id': 'req-08bbf2bd-42ea-433d-951a-a1b722fc026a'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:39.088100 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-08bbf2bd-42ea-433d-951a-a1b722fc026a admin admin] The token cache is being invalidate because project 97074b588da542718abb3ea53aed39df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:57:39.088387 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-08bbf2bd-42ea-433d-951a-a1b722fc026a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:39.089011 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-08bbf2bd-42ea-433d-951a-a1b722fc026a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:39.090844 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1456/2921] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:57:38 2022] DELETE /identity/v3/projects/97074b588da542718abb3ea53aed39df => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:39.095212 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e95f77-6723-4c76-9550-f3ba02059a14 service neutron] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:39.104693 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e95f77-6723-4c76-9550-f3ba02059a14 service neutron] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:39.160558 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba38ba3-f330-4a1a-8129-e93704540933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:39.161430 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba38ba3-f330-4a1a-8129-e93704540933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:39.161711 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba38ba3-f330-4a1a-8129-e93704540933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:39.161987 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba38ba3-f330-4a1a-8129-e93704540933 admin admin] PATH_INFO: `/v3/users/38943e721ed54818b9844d8c4ffeea24` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:39.163219 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba38ba3-f330-4a1a-8129-e93704540933 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38943e721ed54818b9844d8c4ffeea24)` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:39.165131 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba38ba3-f330-4a1a-8129-e93704540933 admin admin] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:39.329826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1ba38ba3-f330-4a1a-8129-e93704540933 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38943e721ed54818b9844d8c4ffeea24'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:39.330409 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1ba38ba3-f330-4a1a-8129-e93704540933 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38943e721ed54818b9844d8c4ffeea24'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:39.331014 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1ba38ba3-f330-4a1a-8129-e93704540933 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38943e721ed54818b9844d8c4ffeea24'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:39.342498 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1ba38ba3-f330-4a1a-8129-e93704540933 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38943e721ed54818b9844d8c4ffeea24'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:39.349745 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.notifications [None req-1ba38ba3-f330-4a1a-8129-e93704540933 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38943e721ed54818b9844d8c4ffeea24'} {{(pid=80515) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:39.408276 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1466/2922] 10.222.0.65 () {58 vars in 1223 bytes} [Tue May 10 20:57:39 2022] DELETE /identity/v3/users/38943e721ed54818b9844d8c4ffeea24 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 20:57:39.804152 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbf2bd-42ea-433d-951a-a1b722fc026a admin admin] Authenticating user token {{(pid=80516) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 20:57:39.805569 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.fernet_utils [None req-08bbf2bd-42ea-433d-951a-a1b722fc026a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80516) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 20:57:39.819929 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbf2bd-42ea-433d-951a-a1b722fc026a admin admin] Validating token access rules against request {{(pid=80516) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 20:57:39.875815 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681c796-bf34-4e4d-8d1f-86dd5ad5c479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '729fa49ea5f943b2b09c35b636e34ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '475f061ace89472aa69923909a044007', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80516) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 20:57:39.876607 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681c796-bf34-4e4d-8d1f-86dd5ad5c479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 20:57:39.876891 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681c796-bf34-4e4d-8d1f-86dd5ad5c479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 20:57:39.877161 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681c796-bf34-4e4d-8d1f-86dd5ad5c479 admin admin] PATH_INFO: `/v3/projects/37af467a93fd40df9b88069558af7e4a` {{(pid=80516) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 20:57:39.878476 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681c796-bf34-4e4d-8d1f-86dd5ad5c479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37af467a93fd40df9b88069558af7e4a)` {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 20:57:39.880432 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681c796-bf34-4e4d-8d1f-86dd5ad5c479 admin admin] RBAC: Authorization granted {{(pid=80516) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 20:57:39.996811 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4681c796-bf34-4e4d-8d1f-86dd5ad5c479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37af467a93fd40df9b88069558af7e4a', 'request_id': 'req-4681c796-bf34-4e4d-8d1f-86dd5ad5c479'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:40.065659 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4681c796-bf34-4e4d-8d1f-86dd5ad5c479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37af467a93fd40df9b88069558af7e4a', 'request_id': 'req-4681c796-bf34-4e4d-8d1f-86dd5ad5c479'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:40.071087 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4681c796-bf34-4e4d-8d1f-86dd5ad5c479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37af467a93fd40df9b88069558af7e4a', 'request_id': 'req-4681c796-bf34-4e4d-8d1f-86dd5ad5c479'} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:40.082450 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4681c796-bf34-4e4d-8d1f-86dd5ad5c479 admin admin] The token cache is being invalidate because project 37af467a93fd40df9b88069558af7e4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80516) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 20:57:40.082735 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4681c796-bf34-4e4d-8d1f-86dd5ad5c479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:40.083363 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: DEBUG keystone.notifications [None req-4681c796-bf34-4e4d-8d1f-86dd5ad5c479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80516) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 20:57:40.084900 dv-cinder-828805-2-iscsi devstack@keystone.service[80516]: [pid: 80516|app: 0|req: 1457/2923] 10.222.0.65 () {58 vars in 1232 bytes} [Tue May 10 20:57:39 2022] DELETE /identity/v3/projects/37af467a93fd40df9b88069558af7e4a => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 21:01:20.333247 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba38ba3-f330-4a1a-8129-e93704540933 admin admin] Authenticating user token {{(pid=80515) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 21:01:20.334656 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-1ba38ba3-f330-4a1a-8129-e93704540933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 21:01:20.467862 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba38ba3-f330-4a1a-8129-e93704540933 admin admin] Validating token access rules against request {{(pid=80515) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 21:01:20.525042 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788579c-c43e-4041-b1c7-df3cdd82c726 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022093dea10d46478b3016395c712d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a006477d1df94390b599621446bd0f02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80515) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 21:01:20.525690 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788579c-c43e-4041-b1c7-df3cdd82c726 service placement] REQUEST_METHOD: `GET` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 21:01:20.525974 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788579c-c43e-4041-b1c7-df3cdd82c726 service placement] SCRIPT_NAME: `/identity` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 21:01:20.526286 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788579c-c43e-4041-b1c7-df3cdd82c726 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80515) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 21:01:20.528106 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.fernet_utils [None req-7788579c-c43e-4041-b1c7-df3cdd82c726 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80515) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 21:01:20.539269 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788579c-c43e-4041-b1c7-df3cdd82c726 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 21:01:20.540826 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788579c-c43e-4041-b1c7-df3cdd82c726 service placement] RBAC: Authorization granted {{(pid=80515) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 21:01:20.550499 dv-cinder-828805-2-iscsi devstack@keystone.service[80515]: [pid: 80515|app: 0|req: 1467/2924] 10.222.0.65 () {60 vars in 1365 bytes} [Tue May 10 21:01:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)